SemAnt

bioguid.info

2007-03-22T13:57:00.000Z

As announced on nodalpoint, I've put together a web site called bioguid.info which, rather grandly, is an attempt to bootstrap the biodiversity Semantic Web by providing resolvable URIs for biological objects, such as publications, taxonomic names, nucleotide sequences, and specimens. These URIs (or "GUIDs") can be resolved by a web browser to display HTML, but under the hood are resolved to RDF (which you can see by viewing the source of the web page you get for a URI).

More 3store3

2007-03-13T10:07:00.000Z

Had a little fun with 3store3 on Fedora Core 4, which is what I run on my servers. 3store3 builds easily on Fedora (in constrast to the fun had building 3store3 on a Mac. However, any SPARQL query beyond a simple listing of all triples just bailed out with scary errors. For example:


ts-explain "PREFIX dc:  
 SELECT DISTINCT ?s  ?o WHERE { ?s  
dc:title ?o . } LIMIT 10"
SQL error 2013: Lost connection to MySQL server during query at util.c:55
SQL error 2002: Can't connect to local MySQL server through socket  
'/var/lib/mysql/mysql.sock' (111) at query.c:316
Warning cannot calculate complexity
Complexity: 0
SELECT DISTINCT v0.lexical AS `s`, v1.lexical AS `o`, v1.datatype AS  
o_dt, v1.language AS o_lang
FROM (SELECT DISTINCT t0.object AS `o`, t0.subject AS `s`
FROM triples t0
WHERE t0.predicate=-8024650864867163606
LIMIT 10) AS `tmp0_202a`, symbols v0, symbols v1
WHERE tmp0_202a.s=v0.hash && tmp0_202a.o=v1.hash
LIMIT 10;

After much cursing, rebuilding,trying different versions of 3store3 and the Redland libraries, gave up and contemplated a crude hack. The SQL query itself works fine in MySQL, but not when 3store3 tries to call MySQL. But, before writing a crude hack I mailed the 3store3 mailing list this morning, and by 9:30 the same morning Steve Harris had given me the solution. I was running MySQL version 4.1.11, and according to Steve there's a bug in early verisons of MySQL 4. He recommended upgrading, so off to do yum install mysql. Sure enough, I get MySQL 4.1.20, and everything works. Yay!

Bio2RDF

2007-03-06T07:30:00.000Z

Stumbled across Bio2RDF by François Belleau. Here's the blurb:

It is now possible to query Bio2RDF triple store using Sesame's SeRQL query engine. The actual triple store contains all annotations about human and mouse from UniProt, Affymetrix and GeneID. It also contains all GO term definitions and OMIM disease description. It contains 50 millions triples and the native RDF store in sesame weigths 3 Go. Watch for the speed of it ! Thanks for the Sesame team for their great work.

There's a blog, and a SourceForge project. The Bio2RDF website itself displays RDF as tables. It doesn't have a query interface but it's a bit hidden. Try here, based on this blog post.
It uses Seasame, JSP, and SeRQL -- not my favourite technologies -- but is an example of people thinking about RDF and triple stores, and actually making stuff. Must polish up my creaking ant demo and play some more with this.
On the subject of ants, Terry Catapano has been playing with Simile, and come up with this demo.

303 and concept URIs - towards GUIDs for all

2007-02-23T05:00:00.000Z

I've bookmarked some stuff relevant to the issue of what HTTP URIs identify. This concerns whether a URI identifiers something in the real world (or a concept), or a document. The "303" refers to the HTTP code returned if the URI is a concept, not a document. The whole topic is a mess, but I need to get to grips with it as I'm working on a service to assign URIs to just about any biologically interesting object. Sigh.

Harvesting handles

2007-02-14T18:19:00.000Z

Finally discovered how to get metadata from DSpace-hosted items, such as the AMNH's Scientific Publications. They have an OAI interface, so that the URL http://digitallibrary.amnh.org/dspace-oai/request?
verb=GetRecord
&metadataPrefix=oai_dc
&identifier=oai:digitallibrary.amnh.org:2246/1999 will retrieve XML metadata for the record hdl:2246/1999 -- in this case, number 2578 of the American Museum novitates.

Oh, and the Simile project already has a OAI to RDF XSLT stylesheet (found courtesy of Leigh Dodds OAI bookmarks on del.icio.us).

Almost forgot, I stumbled across this information at Celestial.

ANeT - an invisible ant resource

2007-01-30T14:08:00.000Z

ANeT is a network to promote ant reserach in Asia. It has some interesting things, although it's a shocking example of poor design. Almost all the text on the home page is not plain text but text written in a GIF image, which means search engines like Google will have a tough time indexing the page, which in turn means it will be hard to find.

And if you can't be found by Google, you don't exist...

Searching Hymenoptera Name Server literature

2007-01-24T18:40:00.000Z

These are some notes on efforts to make the Hymenoptera Name Server literature data base searchable. This work builds on LSID stuff I did earlier, and is also a response to the TAXACOM thread started by Roger Hyam. Donat Agosti has also been requesting something along these lines.

The first step is to suck all the records off HNS, and convert them to RIS format. I then want to import that in to an instance of MyPHPBib (an old project of mine languishing on SourceForge), which gives me a MySQL database of the literature to play with. What I'd like is an OpenURL style search interface that can be used to return records matching a user query.

Notes to self. Character encoding is a major, major pain. I'm running the script on a Fedora Core 4 box as Mac OS X drove me nuts, I'm ensuring that the XML style sheet outputs ISO-8859-1 encoding (to match that returned by HNS), and I set the Terminal character encoding to Western(ISO-8859-1) as well.

Tales of a Semantic Web Consultancy » Blog Archive » 2 papers worth reviewing from ISWC2006

2006-12-01T20:31:00.000Z

From Tales of a Semantic Web Consultancy is
this post on 2 papers worth reviewing from ISWC2006. The Pathetic Fallacy of RDF is a nice summary of why graphs, appealing as they are, aren't the way to think about displaying RDF. Other approaches seem more promising...

Damn DiGIR

2006-11-30T12:07:00.000Z

I've been playing with DiGIR providers, retrieving records to be massaged into RDF as part of a project to aggregate specimens, sequences, publications, and phylogenies. DiGIR was a major advance on what went before (i.e., basically nothing), but in addition to variations in the schema, and the fact that a good portion of the providers are off line at any one time (see Perils of Federation), I'm coming up against the wide range of ways people have of writing dates.

Ideally, I'd like dates in the variation of the ISO 8601 YYYY-MM-DD format described by the W3C W3C, and recommended by Dublin Core. What we can get in DiGIR records is all manner of formats, such as (with specimen codes following)


11/28/2000 12:28:30 PM [KU 195138]
2004-02-20 00:00:00 [MVZ 149006]
2006-08-21 [FMNH 145699]
12/Jun/1983 [KU 195138]
29 Jun 1974 [MVZ 149006]

Now, variation in the <darwin:VerbatimCollectingDate> tag (the last two dates) is expected, but for a computer generated field such as <darwin:DateLastModified> this is a bit much.

Since I do most of my harvesting in Perl, I came across Date::Manip, which manages to convert these into a sensible form (for example 11/28/2000 12:28:30 PM becomes 2000-11-28T12:28:30).

Integration is not easy...

More GUIDs

2006-11-22T13:57:00.000Z

Idylly Googling, I stumbled across The Knowledge Bank at OSU, a DSpace server at Ohio State University. A number of publications on ants are listed there, complete with PDFs. For example, "The Mating Activities of the Ant Myrmica Americana Weber" by Kannowski and Kannowski, published in The Ohio Journal of Science in 1957. This paper has a GUID in the form of a Handle (hdl:1811/4489). What I find interesting is that digitisation efforts by libraries are putting biodiversity literature online as part of a broader effort (i.e., we get these papers, and a GUID, "for free"). This also raises issues about duplication of effort — clearly, if a library has put a PDF online, we don't want to duplicate this. Hence, we need a simple way of finding whether a paper has already been digitised. Google Scholar may be useful for this, although in this case Google finds the paper but Google Scholar doesn't.

Copyright on images

2006-11-20T18:23:00.000Z

This post is a copy of a comment I wrote on The Ant Room post Synchronizing and Copyrighting Images, which I've repeated here so I don't loose track of it.

Two thoughts on copyrighting images. The first is why choose copyright © as opposed to a Creative Common (cc) license? With a cc license you get to specify what I can and can't do with the image, without me having to ask you. By sticking "© K. T. Ryder Wilkie 2005" on an image (e.g., your gorgeous picture of Acanthoponera peruviana), I then have to contact you to ask your permission. For one or two images, that's OK I guess, but what it I want to use lots of images? What if you are on holiday?

The second comment is that I can read "© K. T. Ryder Wilkie 2005" but computers can't (at least, not easily). There other other ways to tag images that computers can read this information. Examples include EXIF tags (as used by Antweb, as mentioned on my iSpecies blog) which get embedded in the image file itself (also XMP information added by Photoshop, or Flickr tags (for example, this image of Strumigenys precava). My point is that if people are going to make use of your work on a large scale, using Creative Common licenses and embedding that information electronically in the image in the form of metadata will make your hard work even more useful.

If sharing information on biodiversity is going to take off, then we need to start thinking about how to share, and how to make our information accessible to computers, not just people.

SVG specimen maps from SPARQL results

2006-11-11T18:58:00.000Z

One reason why I've put off adding specimen maps to iSpecies, despite repeated requests, is that Google Maps (my preferred mapping tool) is slow if you have lots of specimen records. I've played with some other tools, notably Map Bureau's Flash-based pointMapper, but what I'd really like is a quick and simple way to display a bunch of specimen records. Because the same issue comes up with SemAnt, I thought it's time to do something about it. I stress that I really like Google Maps, but for some purposes it's overkill. Furthermore, loading hundreds of points will take too long.

So, the idea is to take georeferenced specimen records and put them on a map of the world. Slowly it dawned on me that this was trivially easy. Firstly, take a map of the world drawn using the equirectangular (or plate carrée) projection (Wikipedia provided the example below).

This projection a simple connection between geographic location and pixel position. For example, if the map is scaled to 180 pixels high and 360 pixels wide, then you have a 1 pixel/degree grid. Hence, plotting localities is no harder than plotting a X-Y scatter plot.

Now, all I need to do is take a SPARQL result with latitude and longitudes and draw the localities on this map. One way to do this is to draw the points using SVG, so I can use a XSL transformation to generate the map. If I wanted to support zooming then ideally I'd have the map itself in SVG, but I just want a small world map, so I "cheat" and use a bitmap as the base map. This can be included like this:


<image x="0" y="0" width="360" height="180" 
xlink:href="http://...360px-Equirectangular-projection.jpg" />

The trick is to convert latitude and longitude to coordinates on the bitmap. For example, specimen casent0008682-d03 of Melissotarsus emeryi was collected from 31°58'0'' S, 18°51'0'' E, which in decimal values is latitude -31.966667, longitude 18.85. Now, how do I convert these values into a location on a 360 × 180 image? In SVG the coordinates grow from the upper left, whereas on the map shown above 0,0 is in the centre, such that southern latitudes are negative, as are western longitudes. We can use a transform to move the origin of the x- and y-axes to the left 180 pixels, and down 90 pixels, so that the origin of the graph is the intersection of the equator and Greenwich Meridian. We also have to invert the y-axis because in SVG it goes from top to bottom. This diagram shows the difference between SVG and geographical coordinates:

This transformation is achieved by this statement


<g transform="translate(180,90) scale(1,-1)" >
.
.
.
</g>

This idea came from hack #55 in Michael Fitzgerald's book XML Hacks. Here is the XSLT I use.


<?xml version='1.0'?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:res="http://www.w3.org/2005/sparql-results#" xmlns="http://www.w3.org/2000/svg"
    xmlns:xlink="http://www.w3.org/1999/xlink" exclude-result-prefixes="res xsl">
    <xsl:output method="xml" version="1.0" indent="yes"/>
    <xsl:template match="/">
        <svg>
            <xsl:attribute name="width">360px</xsl:attribute>
            <xsl:attribute name="height">180px</xsl:attribute>
            <rect id="dot" x="0" y="0" width="4" height="4" style="stroke:none; stroke-width:1; fill:solid"/>
            <image x="0" y="0" width="360" height="180" xlink:href="http://upload.wikimedia.org/wikipedia/commons/thumb/e/ea/Equirectangular-projection.jpg/360px-Equirectangular-projection.jpg"/>
            <g transform="translate(180,90) scale(1,-1)">
                <xsl:apply-templates select="//res:result"/>
            </g>
        </svg>
    </xsl:template>
    <xsl:template match="//res:result">
        <use xlink:href="#dot">
            <xsl:attribute name="transform">
                <xsl:text>translate(</xsl:text>
                <xsl:value-of select="res:binding[@name='long']/res:literal"/>
                <xsl:text>,</xsl:text>
                <xsl:value-of select="res:binding[@name='lat']/res:literal"/>
                <xsl:text>)</xsl:text>
            </xsl:attribute>
        </use>
    </xsl:template>
</xsl:stylesheet>

This transforms a SPARQL result that looks something like this:


<result>
   <binding name="lat"><literal>10.266666</literal></binding>
   <binding name="long"><literal>-84.083336</literal></binding>
</result>

One thing which drove me nuts for a while was that the SVG rendered fine in Safari using Adobe's plugin, but not in Camino, which uses the same rendering engine as Mozilla. Turns out Camino needs http://www.w3.org/2000/svg to be the default namespace, so xmlns="http://www.w3.org/2000/svg" is fine, but it barfs over xmlns:svg="http://www.w3.org/2000/svg". Sigh.

Here is an example SVG file rendered using the XSLT style sheet, but using a different background map, showing the distribution of the ant Azteca constructor. The source SVG is here.

A nice, simple map, with minimal effort.

Perils of federation

2006-11-08T11:50:00.000Z

Dave Vieglais gave what looks like an interesting presentation at TDWG 2006. BigDig monitors the status of DiGIR providers that serve museum specimen records. I've not managed to get the map background to appear, but here's a snapshot of the geographical distribution of providers, and their status:

The happy faces are DiGIR providers that are live, the sad faces are not responding. What is interesting that a fair chunk are offline, of the 180 registered providers, 25 have never responded, and there are something like 17 variations of the DiGIR schema out there.

It's a little scary that so many providers are offline, and that they differ in the format of the messages they accept and return. For federated searches that are "live," this spells disaster. His presentation is here (rather unhelpfully in Open Office format, so I've made a PDF).

Automatically growing an ant bibliography

2006-10-23T18:30:00.000+01:00

Earlier on iPhylo I'd mentioned the issue of updating a triple store of ants, or indeed, any data base. As an experiment, I've put together a Perl script that can be used to update a data base in Connotea with recent papers on ants. The script makes of a number of web services, and uBio's RSS feeds. It does the following:

Takes an RSS feed for Formicidae from uBio. This feed lists recent papers on ants, as identified using uBio's taxonomic name recognition algorithms.

Extracts DOIs or PubMed identifiers from the RSS feed. If a DOI isn't found, I see if we can extract one from the <link> tag (typically a URL to the article). uBio does a pretty good job of getting DOIs, but misses some (e.g., for Blackwell and BioOne journals).

Extracts taxonomic names from the content of the <title> and <description> tags using a SOAP call to uBio's FindIT web service. Ideally, uBio would do this for us, since it has already parsed the journal feed, but for now I do it.

Uses Yahoo's term extraction web service to extract keywords

Submit the article GUID (DOI or PubMed id), and the tags to Connotea using the web API.

Here's a sketch of the process.

The papers are stored in my semant library. Because it is entirely automated, it could be run regularly (as a cron job, say) to update the library, hence the list of ant papers would grow without any human intervention. At the same, however, users with access to the semant library could manually edit the tags if they feel Yahoo and uBio have missed some relevant terms.

Note also that names recognised by uBio are tagged with LSIDs for the names as well, which means we could resolve those to RDF. In the same way, the Connotea data base itself can serve RDF (here are the ant papers in RDF). Hence, we could easily populate a triple store with metadata about papers and names.

What I like about this script is that it brings together a number of themes.

GUIDs play a key role here. Connotea knows which papers uBio has extracted by using the DOI (or PubMed identifier). Not only does this enable Connotea to know which paper I want, but it uses that identifier to extract metadata about the paper, for example via CrossRef. It also knows whether any other user has already added that paper.

Web services mean that I don't have to reinvent the wheel. If I want to pick out taxonomic names, I use uBio. To extract keywords for tagging, I use Yahoo. To store data, I use Connotea's API.

Tagging makes it easy to add information to a reference.

Social networking through using an open database like Connotea. People can discover other people's libraries through shared papers or shared tags.

RSS pops up at the start and at the end. The whole process starts with a RSS feed (itself an aggregation of numerous journal RSS feeds), and the resulting Connotea data base serves RSS, so others can readily make use of the results.

Geonames ontology in OWL - GSWB

2006-10-17T19:42:00.000+01:00

Via Danny Ayers' blog, I found this discussion of the
Geonames ontology in OWL - GSWB, which is described in full at Geonames. What is nice about this is both that geographic localities get URIs that resolve to RDF (e.g., http://ws.geonames.org/rdf?geonameId=3020251), but also there is an ontology specifying the relationships amongst geographical features. Perhaps another good opportunity for biodiversity informatics to reuse, rather than reinvent.

(Via Danny Ayers.)

Formicidae on Flickr

2006-10-03T19:54:00.000+01:00

Simon Rycroft wrote a nice script to pull all the ant pictures off Flickr. Makes for an interesting image.

Ants on Flickr

2006-10-02T15:35:00.000+01:00

More an insight into how I waste my life than anything else, but saw this post and followed the link to JochenB's set of ant photos on Flickr, which includes some stunning pictures of ants, such as this one of Eciton burchelii (see original here).

Kewl. Now, we want to get this into a triple store, so I need to play with the Flickr API to convert it to RDF, perhaps using Flickr photo into RDF as a starting point. Note that we get a wealth of tags, plus information on licensing from the Flickr record.

Why can't we spell!?

2006-09-29T18:19:00.000+01:00

I know I'm not a great speller, but it gets frustrating when you discover how many potentially useful links to information are broken due to typos. For example, I stumbled across this page, which states that LACM ENT 164470 is the type specimen of Apterostigma gonides. Hmmm ... problem is, there is no such species. What they meant was Apterostigma goniodes (note the missing "o"). The fact that the only Google hits for "Apterostigma gonides" is the LACM page itself is a clue that something's up.

So, a potentially useful resource listing types housed at the Natural History Museum of Los Angeles County looses value through a typo. Argh!

Organizing the Ant Internet - from The Ant Room

2006-09-28T16:50:00.000+01:00

From The Ant Room:

Organizing the Ant Internet
This is what I am thinking about today -- I have this organization thing. I have a very strong need to have things in my life organized. A big pile of papers and junk drives me crazy. I just want to go through them all and put them into categories and file them away or throw away the trash and make everything look nice and neat. I get this feeling sometimes when I am browsing through the internet and looking at ant sites. I want to take them all off the web, look at them, clean them up a bit, throw away the junk, and put them all together in one well-organized drawer. There are so many ant sites nowadays and each and everyone seems to want to have everything you could ever want from an ant site, but none of them do. And I just think, if they could all get together, you really would have the best ant site ever. I'm not really sure why they don't. Even just a little bit of information sharing would be helpful. For instance, you've got AntWeb, which is a fabulous website if you are hoping to look up ants from Madagascar, but not if you are hoping to look up ants from Costa Rica. Why is that? The ants of Costa Rica have a fabulous webpage. It doesn't seem like it would be that difficult to import all of those costa rican ants onto AntWeb. DiscoverLife has done it. They don't have any checklists from Australia or Japan, though, which are also two groups of ant fauna with great webpages. Shouldn't we be trying to incorporate all of this information together? Even just a link to the other websites would be nice. It took me forever to figure out where the good websites were. I don't even trust the lists that are on DiscoverLife now -- I have a list of Tiputini ants on Discoverlife -- it is terribly out of date and I can't figure out how to update the list so I've just let it go. Tree of Life is another webpage that is basically useless to me. There are these beautiful photos but when you get down to the species level, you get a statement like "127 described species" but no actual list of species. Pseudomyrmex, for instance, has no species list on the Tree of Life website. Why not? A list certainly exists. And there are even labs that have been looking at this genus for years. Cephalotes, Procryptocerus, the Attini, Megalomyrmex, Pheidole, and Dolichoderus -- all genera that have no list of species on their tree of life webpage. Why hasn't someone added more ant information to this website? Or other websites? It is very frustrating to me. I wish someone would put me in charge of making one fantastic ant webpage that would incorporate everything. I know it would drive me insane but it would be very satisfying work. [The Ant Room]

I guess this is one of the motivations behind SemAnt -- exploring how to integrate diverse resources on ants into a single framework.

More on Wikis

2006-09-25T13:21:00.000+01:00

Who writes Wikipedia?

That's the title of a longish, well-argued, very readable piece by Aaron Swartz, the highlight for me being a conclusion he arrived at by mining the data:

When you put it all together, the story become clear: an outsider makes one edit to add a chunk of information, then insiders make several edits tweaking and reformatting it. In addition, insiders rack up thousands of edits doing things like changing the name of a category across the entire site -- the kind of thing only insiders deeply care about. As a result, insiders account for the vast majority of the edits. But it's the outsiders who provide nearly all of the content.

(My emphasis). This contrasts with the results of some shallower research done by Jimbo Wales, and this kind of thing is why I for one would like to see Aaron on the Wikipedia board (although I couldn't vote because I've done <400 edits).

[Raw]

Adding triples using EditGrid

2006-09-20T14:12:00.000+01:00

Vince Smith has constantly been telling me that for many biologists, "database" means an Excel spreadsheet, and that a big problem is simply getting data into a form that can be used online. Bearing that in mind, and also mindful of how much data is kicking around that isn't in "real" databases, I've been playing with EditGrid as a tool for adding triples to a triple store. I've commented on EditGrid elsewhere in the context of collaborative data matrices.

So, here's the situation. In my triple store I have information on ant specimen INBIOCRI001284215, obtained from AntWeb. Now, AntWeb has no pictures of this specimen. However, John Longino's pages on Acromyrmex coronatus include pictures of this specimen. How do I get that information into my triple store, without writing RDF?

One approach is to create a spreadsheet with three columns (subject, predicate, object), and create the triples, one per row. Now, I could just do this on my computer using, say, Excel, but that's not nearly cool enough, so I'll use EditGrid. But seriously, I'm going to use EditGrid because:

You can see it, whereas you can't see a file on my computer
You and I could collaborate on editing the data in EditGrid

The spreadsheet contains triples, such as these:

subject	predicate	object
http://www.evergreen.edu/ ... /INBIOCRI001284215_face_orig.jpg	foaf:depicts	http://www.antweb.org/ ... inbiocri001284215

In this case the subject and the object are represented by URIs (here they are URLs, but they could also be LSIDs or DOIs). You can see the complete spreadsheet here. The triples link the picture to the specimen, tell us that http://www.evergreen.edu/ants/genera/acromyrmex/species/coronatus/INBIOCRI001284215_face_orig.jpg is a picture (dc:type image), that the picture has a thumbnail, and is of Acromyrmex coronatus. Armed with these triples, I can now find a picture of this ant in my triple store.

Fine so far, but how do we get this into the triple store I hear you ask? EditGrid's permalink feature can export the spreadsheet in a range of formats, including XML. So, what I do is grab the XML, apply a XSL style sheet to convert it to RDF, then import the resulting RDF into the triple store. The key thing is once the data is in the spreadsheet, the rest is trivial. Here's the XSL style sheet. It has limitations, notably the assumption that URIs are URLs.


<?xml version='1.0' encoding='iso-8859-1'?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" 
    xmlns:foaf="http://xmlns.com/foaf/0.1/" 
    xmlns:dc="http://purl.org/dc/elements/1.1/">
    <xsl:output method="xml" version="1.0" encoding="iso-8859-1" indent="yes"/>
    <xsl:template match="workbook">
        <rdf:RDF>
            <xsl:apply-templates select="//row"/>
        </rdf:RDF>
    </xsl:template>
    <xsl:template match="row">
        <xsl:if test="@row != '0'">
            <xsl:element name="rdf:Description">
                <xsl:attribute name="rdf:about">
                    <xsl:value-of select="cell[1]/@input"/>
                </xsl:attribute>
                <xsl:variable name="predicate" select="cell[2]/@input"/>
                <xsl:variable name="object" select="cell[3]/@input"/>
                <xsl:choose>
                    <xsl:when test="contains($object, 'http://')">
                        <xsl:element name="{$predicate}">
                            <xsl:attribute name="rdf:resource">
                                <xsl:value-of select="$object"/>
                            </xsl:attribute>
                        </xsl:element>
                    </xsl:when>
                    <xsl:otherwise>
                        <xsl:element name="{$predicate}">
                            <xsl:value-of select="$object"/>
                        </xsl:element>
                    </xsl:otherwise>
                </xsl:choose>
            </xsl:element>
        </xsl:if>
    </xsl:template>
</xsl:stylesheet>

This particular spreadsheet makes some assumptions of the user, namely that they can figure out what is the subject and what is the object, and are comfortable choosing predicates. However, being collaborative, others could help out by editing the spreadsheet. Furthermore, one could create spreadsheets that aren't quite so complicated, and aren't geared towards the developer. For example, one basic source of information I'd like to capture is geographic location, and there is probably a lot more information available in papers than in georeferenced museum collections. Hence, a spreadsheet like this

observation	lat	long
locality	-34.0	156.26

could be used to capture locality information, and would require minimal effort to convert into RDF. We'd just have to modify the XSL style sheet shown above.

The key point of all of this is that with minimal effort we can capture information that is not in the triple store, and we can make it eas(ish) for people with data to contribute. Given that EditGrid can import Excel files, somebody interested in sharing their data could do the grunt work in Excel on their own computer, then move everything to EditGrid, which makes it accessible to others.

Simple and open wins...

Connotea tags

2006-09-07T17:49:00.000+01:00

The following SPARQL query returns the "tags" for a Connotea reference using the DOI as the search term (in this case doi:10.1007/bf02224026):


PREFIX dc: <http://purl.org/dc/elements/1.1/>
SELECT ?subject
WHERE {
  ?doi ?bnode 'doi:10.1007/bf02224026'
. ?connoteaURI ?identifier ?doi 
. ?item ?connotea ?connoteaURI
. ?item dc:subject ?subject
}

The graph being queried is this RSS file, which I've put in a triple store.
This query is simply following the path in the RDF from the DOI <connotea:idValue>10.1007/bf02224026</connotea:idValue>.

Note: One potential "gotcha" is that DOI's are not case sensitive, but SPARQL queries are (oh oh).

GenBank extras

2006-09-06T00:27:00.000+01:00

Idly playing with ants, it is time to blog two things that come up a few times. The first is that GenBank has links to literature that could do with updating. For example, the sequence U11912 from the fungus "Atta mexicana symbiont JF-1" is listed as being published in


  AUTHORS   Rehner,S.A., Chapela,I.H., Schultz,T.R. and Mueller,U.G.
  TITLE     Evolutionary history of the symbiosis between fungus-growing ants
            and their fungi
  JOURNAL   Unpublished

Well, this was published in Science (doi:10.1126/science.266.5191.1691) in 1994. The DOI seems broken (sigh), so here is a direct link. Ulrich Mueller's web site has a link to the PDF.
The other point is that searching the nucleotide database for "Atta mexicana" turns up no ants, but the above mentioned fungus. We get the hit because there is a line in the GenBank record that lists the ant host.


/specific_host="Atta mexicana"

As I've mentioned over on the iSpecies blog, GenBank records often contain this sort of useful information. Hence, we could search for ants and extract information about their fungal associates.

LSIDs and DOIs for ant (and other hymenopteran) literature

2006-08-31T20:23:00.000+01:00

I've now got LSIDs for literature served by the Hymenoptera Name Server and Antbase. As an added bonus, I serve DOIs where they exist. The RDF metadata for the LSIDs are generated on the fly by querying the Hymenoptera Name Server using a simple XML service Norm Johnson provided. Given a URL of the form http://atbi.biosci.ohio-state.edu:210/hymenoptera/manage_lit.ris2xml?id= the service returns a XML document for the corresponding reference. I simply transform this into RDF.
However, as an added feature, if the publication is an article I use Crossref's OpenURL resolver to see if a DOI exists for the publication -- if so, this gets added to the RDF as a <dc:identifier> tag.
Like most other LSIDs that I serve, the service is slower than it could be because everything is done on the fly, and in this case two calls may be needed to generate the metadata (see diagram below). The authority is antbase.org.lsid.zoology.gla.ac.uk, and the namespace is pub. Here is an example LSID, which you can either click on in your browser (urn:lsid:antbase.org.lsid.zoology.gla.ac.uk:pub:3080) or using Launchpad (urn:lsid:antbase.org.lsid.zoology.gla.ac.uk:pub:3080)

One encouraging observation is that DOIs are not restricted to recent literature. For example, urn:lsid:antbase.org.lsid.zoology.gla.ac.uk:pub:3080 dates from 1952, but has a DOI (doi:10.2307/2422200).

Wikis

2006-08-28T23:05:00.001+01:00

Triple stores containing specimens, sequences, images, literature, etc. are all very well, but there is a lot of information that is not captured by such a system. For human users (as opposed to dumb computers), often a simple summary is more informative than a set of images and a map, especially if that summary mentions something interesting, such as why the "Google ant" was so named, or that the trap-jaw ant Odontomachus bauri has incredibly fast jaws (doi:10.1073/pnas.0604290103). There is also a lot of information that may one day be semantically encoded, but for now will only be captured as text (such as extensive commentaries on web sites).

Lastly, there is the issue of getting people involved. One of the striking things about biodiversity web sites is the lack of community involvement. My question is "why is this the case?" Here are some thoughts:

If feedback consists of sending an email to the person/organisation running the site, then there's little incentive to get involved. Why bother typing a detailed summary of why a particular piece of information is wrong, only to have that essay disappear into a black hole?

Annotations can be valuable, but there is an issue of trust. Why invest effort in a site that may be a short-lived toy? There are so many sites competing for attention.

One way to foster involvement might be through a Wiki, whereby anybody can contribute to writing a page about each organism. However, biodiversity Wikis such as Wikispecies have, in my opinion, been a spectacular failure, as evidenced by the number of missing pages or stubs. Perhaps part of the reason is the lack of content, which could be addressed by pre-populating each page with basic information from a database (such as name, any specimens, images, literature, etc.). In other words, each page would start with the level of detail of an iSpecies report (for background to iSpecies visit my iSpecies blog). As Kevin Kelly commented at the recent Google Sci Foo camp, people are much more likely to edit existing content than create content de novo.

Wikis are all very well, but my major worry is the potential to loose information. Here's the problem. Suppose I generate a Wiki page for an ant, and include information on its distribution. What happens if the underlying distribution data changes? Now that the page is in Wiki form, it will be out of date. Furthermore, I'm not sure I want users editing distribution records -- these should really be edited at the level of the source database, so that the changes propagate to other users of those data.

One possibility is to use custom tags in the Wiki. When the HTML page is generated, Wiki tags are rendered in the usual way, but the custom tags are replaced by the results of a database call (for example, a SPARQL query). Hence, something like %DISTRIBUTION would be replaced by a Google map of the specimens for that taxon. This would mean that the distribution map would always reflect the current database, and the user (assuming they don't delete the %DISTRIBUTION tag itself) won't be able to alter that information. Of course, this means we need some mechanism for users to inform the curators of the source data of any potential errors. This is particularly important if we pre-populate the Wiki page with information that may be incorrect (such as images harvested from a search engine).

We could also help things by encouraging standard ways of linking to other resources, or storing data. For example, say a user edits a page and adds a citation to a paper that isn't in the underlying triple store. Ideally we would get that new paper into the triple store (rather than have it languish in the Wiki text). There are some ways to do this, such as extracting metadata from DOIs, and using local links [need to think about this].

Likewise, with images, if we have a convention that images get posted to, say, Flickr, then we have a means for storing metadata about those images directly in our triple store.

Credits
These ideas have partly come out of conversations with Rebecca Shapley at Google, and Dave Thau at the California Academy of Sciences.

So that's how they do it - adding a message to mailto

2006-08-10T18:00:00.000+01:00

The href="mailto:" tag is useful, and I know how to add a subject, but today I learnt how to add a message as well. The example comes from the DOI web site:

This bit of code:

<p>If you believe you have requested a DOI name that should be found, you may report this error to
<a href="mailto:doi-help@doi.org?subject=DOI Not Found&body=The%20following%20DOI%20was%20not%20found:%0D%0A...f">doi-help@doi.org</a>.
<b>Please include information regarding where you found the DOI in your message: http://www.journals.royalsoc.ac.uk/....pdf</b></p>

Gives this result:

If you believe you have requested a DOI name that should be found, you may report this error to
doi-help@doi.org.
Please include information regarding where you found the DOI in your message: http://www.journals.royalsoc.ac.uk/media/d6lntlwywjn3hdu6ekdw/contributions/0/5/8/3/058352377848735w.pdf

The &body= tag gives the text of the email message. Neat.

Taxonomic treatments

2006-07-16T16:18:00.000+01:00

For a demo for Donat Agosti, I've added some taxonomic treatments to iSpecies.org. I'd done this before, but after the server got hacked I didn't restore the treatments because they are served via a triple store, and I hadn't got that running. Now that the triple store is up, I looked at this again. Here's what is involved.

Treatments

TaxonX is a XML mark up for taxonomic descriptions. The idea is to locate and mark blocks of text that describe a taxon. For more details see the AMNH's NSF Taxonomic Literature Projectpages.

Donat has been marking up various ant papers manually as proof of concept, but the process will be automated. I want to be able to serve up a taxonomic description of a name, e.g. "Proceratium google". Because I want everything to be in a triple store, I need to map TaxonX to RDF. Here's what I do:

The URI of the paper is the link to the PDF in AntBase. This should really be something else (LSID, DOI, Handle, PURL), but it will do for now.

Each treatment is extracted from the TaxonX document using XPath. I use a Perl script to pull out each node matching //tax:treatment

Each treatment gets a URI, based on the URI of the paper containing the treatment, and the XPath to the treatment, e.g. http://antbase.org/ants/publications/8538_fisher//tax:treatment[1]. The idea is that one could use the identifier to extract the relevant block of text from the TaxonX XML document (i.e., the identifier would be useful beyond my triple store). Although I worry that this is not semantically opaque, its seems a useful idea, and my worries eased when I discovered that Annotea uses the same idea.

The actual treatment is stored as a block of <![CDATA[..]]>, so the original TaxonX markup is preserved.

Each treatment is linked to the containing paper by the Dublin core term <dcterms:isPartOf>. I also have the inverse link <dcterms:hasPart> to link the publication to the treatments it contains.

I have some minimal metadata about the publication (title, format), and about each treatment (name of taxon stored in <dc:subject>). This is extracted from what is in the TaxonX document - clear TaxonX needs more information on the source.

Each treatment is typed using <dc:type>treatment<dc:type>. I do this so that I can classify results for a query (as part of another project).

So, a publication is modelled like this:

And a treatment is modelled like this.

SPARQL

Currently iSpecies treatments are retrieved using RDQL, but SPARQL is rather nicer. Finding the treatment for a taxon is a simple SPARQL query, e.g.:


PREFIX gla: <urn:lsid:lsid.zoology.gla.ac.uk:predicates:>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX dcterms: <http://purl.org/dc/terms/>
PREFIX tax: <http://research.amnh.org/informatics/taxlit/taxonx/taxonx1>
SELECT ?uri ?publication ?title ?treatment
WHERE {?uri dc:subject 'Proceratium google'
   ?uri gla:treatment ?treatment .
   ?publication dcterms:hasPart ?uri .
   ?publication dc:title ?title
}

Display

To display the results I take the SPARQL XML result, convert the encoded TaxonX block to XML mark up, then apply a simple XSLT style sheet. The results aren't pretty, but it works.

Future directions

As I've mentioned in an earlier post, what I'd really like is to have GUIDs for these publications sorted out, and more mark up. In particular, literature cited, specimens, and other taxonomic names should be marked up so that these links can be extracted. If this is done well, then we could do things like:

Generate distribution maps for papers that don't have maps

Generate synonymies from lists of names

Infer type status even if specimen databases don't have this information

etc.

The trick will transforming TaxonX to RDF.

Currently playing in iTunes: Shelter by Ray LaMontagne

Disconnected databases

2006-07-08T11:13:00.000+01:00

One consequence of having multiple databases is that they can get out of sync, that is, information in one database might not be updated to reflect changes in another. I've touched on this earlier when discussing unidentified ants in GenBank (Discovering new things and Ants, RDF, and triple stores).

I've also come across cases where AntWeb is out of date. For example, the ant Strumigenys rubigus was described in 2000 by Brian Fisher. In the TaxonX marked up version of the original paper (available here), the holotype is listed as:

Holotype worker, Madagascar: Prov. Toamasina, F.C. Andrianantantely, 18 deg. 41.7 min. S, 48 deg. 48.8 min. E, 530 m 4-10.xii.1998, ex rotten log, rainforest, #49-2 (H.J. Ratsirarson) (MCZ).

Now, in an ideal, joined-up world, we'd have a link from the Fisher paper to the actual specimen. A bit of fussing (i.e., searching for "Strumigenys rubigus" on AntWeb) reveals that the holotype is casent0005630. The AntWeb page for this ant has no indication that this is the holotype, although there is a picture of the specimen labels that make it clear that this is what it is.

Having multiple sources of information makes it harder to keep things up to date, which is another reason why I think RDF and triple stores (or distributed queries) will help. So long as we have metadata about the specimen and the publication, we can make the inference that casent0005630 is the holotype of Strumigenys rubigus. This may ease the burden on individual databases. Rather than the curators of AntWeb having to update AntWeb manually every time a new name is published, a portal along the lines of my SemAnt toy could summarise this new information easily, if (and it's a big if) we have the links between publication and specimen.

SPARQL query for classification

2006-07-01T03:28:00.000+01:00

After spidering uBio the next task is how to display the classification of a taxon. The following SPARQL query does the trick:


PREFIX ubio: <urn:lsid:ubio.org:predicates:>
PREFIX gla: <urn:lsid:lsid.zoology.gla.ac.uk:predicates:>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX dc: <http://purl.org/dc/elements/1.1/>

SELECT DISTINCT ?node, ?title, ?rank, ?name, ?description

WHERE {
   <urn:lsid:ubio.org:namebank:2735665> gla:objectiveSynonym ?display .
   ?class ubio:namebankIdentifier ?display .
   ?class ubio:classificationName ?name
   ?class ubio:classificationDescription ?description .
   ?class gla:lineage ?seq .
   ?seq ?li ?node .
   ?node dc:title ?title .
   ?node gla:rank ?rank
}

This query takes a canonical name in uBio, finds the display form, and from that the classification. The lines


   ?class gla:lineage ?seq .
   ?seq ?li ?node .

fetch the lineage which is stored as a sequence:


<gla:lineage>
<rdf:Seq>
<rdf:li rdf:resource="urn:lsid:ubio.org:classificationbank:5178917"/>
<rdf:li rdf:resource="urn:lsid:ubio.org:classificationbank:5095531"/>
.
.
.
<rdf:li rdf:resource="urn:lsid:ubio.org:classificationbank:5131593"/>
</rdf:Seq>
</gla:lineage>

This method of describing a taxonomic lineage was described in my paper in
Biodiversity Informatics
. uBio serves the lineage from lower to higher taxon (i.e., bottom up), but I want to display it top down. I do all display using XSLT style sheets, so we use the <xsl:sort order="descending"> trick (see here for an example). Then it's simply a case of indenting each node using <xsl:value-of select="18 * (position()-1)"/>, and borrowing uBio's end.png to get the tree effect.

Classification

2006-06-30T23:23:00.000+01:00

Displaying a name by itself isn't very useful, so I'm exploring adding a classification to the ant demo. The question is which one? As a trial I've decided to use ITIS, based on the October 12, 2005 dump used by uBio (you can see it here). There are some 15,000 ant taxa in ITIS.

The plan is to retrieve the classification by spidering the uBio site, starting with the RDF for Formicidae in the ITIS classification (urn:lsid:ubio.org:classificationbank:5095531). By following the <ubio:hasChild> tags, we can traverse the complete tree.

One issue is getting my head around uBio's name structure. I use their FindIT SOAP service to get LSIDs for names from NameBank. FindIT returns canonical name ids, but ClassificationBank uses different LSIDs for the names (the "display name"). To give a concrete example, using FindIT to search on "Melissotarsus insularis" yields the LSID urn:lsid:ubio.org:namebank:2735665, whereas in the ITIS classification, the ClassificationBank record (urn:lsid:ubio.org:classificationbank:5184916) links to urn:lsid:ubio.org:namebank:560445.

TreeBASE rocks

2006-06-28T04:32:00.000+01:00

I gave a talk today ("Dude, where's my tree?") at the Evolution 2006 meeting at Stony Brook. It was intended as a somewhat tongue-in-check overview of some issues concerning TreeBASE, and broader areas of biodiversity informatics, making use of ants as an example (see my SemAnt project).
Michael Donoghue took me aside after the talk and made some interesting points. He was a little tired — understandably — of hearing that "TreeBASE sucks" (e.g., my CIPRES talk), and felt that my constantly saying this was counter productive. It could also lead to people not putting their data in TreeBASE because they'd heard that it "sucks".
There is an element of social responsibility here, I guess. I resolutely avoid politics. I don't mean this in a pejorative sense, it's just that I don't have the temperament or skill for it, unlike Michael himself (Lee Belbin is another person in this area who strikes me as a very skilled manager).
Now, my talk was intended to be fun, and I was taking the piss out of myself as much as anything. I also think the things we criticise are the things we value the most. But that said, let be make it clear that TreeBASE is very important. As editor of Systematic Biology I've made authors submit data to it. I have a lot of respect for the work Michael, Bill Piel, and Mike Sanderson put into TreeBASE. If you have phylogenetic data — submit it to TreeBASE. It's the best we have. It's just that, well, as a community we could do better.

Taxonomic names, metadata, and the Semantic Web

2006-06-28T03:46:00.000+01:00

My paper "Taxonomic names, metadata, and the Semantic Web" has appeared in Biodiversity Informatics.

Life Science Identifiers (LSIDs) offer an attractive solution to the problem of globally unique identifiers for digital objects in biology. However, I suggest that in the context of taxonomic names, the most compelling benefit of adopting these identifiers comes from the metadata associated with each LSID. By using existing vocabularies wherever possible, and using a simple vocabulary for taxonomy-specific concepts we can quickly capture the essential information about a taxonomic name in the Resource Description Framework (RDF) format. This opens up the prospect of using technologies developed for the Semantic Web to add ``taxonomic intelligence" to biodiversity databases. This essay explores some of these ideas in the context of providing a taxonomic framework for the phylogenetic database TreeBASE.

Publications via iTunes - kewl!

2006-06-25T03:58:00.000+01:00

Greg Restall has described how he put his papers into the iTunes music store. How cool is that! A nice demonstration of how RSS makes all sorts of interesting applications possible.

(Via All My Eye.)

3Store3

2006-06-20T23:38:00.000+01:00

Building 3Store3 3.0.14 on Mac OS X is an absolute pain. It's pretty clear the developers haven't done so, because there are a slew of dependencies that aren't mentioned. It builds fine on Linux, so it's a case where the developers haven't realised that the assumptions they make on Linux don't always hold on other platforms (such as my beloved Mac).

So, what happens after we type ./configure?

Firstly, we couldn't find rasqal (part of Redland)

configure: error: Package requirements (rasqal >= 0.9.11) were not met:
Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

OK, so edit your .bash_profile to contain these lines:

PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig
:/opt/local/lib/pkgconfig
export PKG_CONFIG_PATH

(/usr/local/lib/pkgconfig is where Redland package config files are stored, /opt/local/lib/pkgconfig is used by Darwin Ports, see below.)

Next, we don't have glib, so it's off to Darwin Ports, which packages a lot of Open Source tools for Mac OS X. Install it, then at the Terminal type:


sudo /opt/local/bin/port install glib2-devel

(this may take a while...). Make sure you have /opt/local/lib/pkgconfig in your PKG_CONFIG_PATH variable (see above). Now, we get


configure: error: Cannot find Berkeley DB library version 4

Sigh. So, we grab Berkeley DB 4, cd to the directory, and


cd build_unix
../dist/configure
make
sudo make install

So we're there, right? Not so fast, did you think this was meant to be easy? 3Store3 assumes Berkeley DB4 is somewhere it ain't.


su
mkdir /usr/include/db4
cd /usr/include/db4
ln -s /usr/local/BerkeleyDB.4.4/include/db.h
./configure LDFLAGS=-L/usr/local/BerkeleyDB.4.4/lib

Are we there yet?! Yes.


sudo make install

To set up the triple store:


ts-setup

Then $£@!#, I discover I need MySQL version 4.1.x (I'm running 4.0.21). To be fair, the documentation (what that) states this pretty clearly. OK, so we move the data safely out of the way, grab 4.1 from www.mysql.com, and install it. I was running CompleteMySQL, so I need to remove that from my path, otherwise we get the wrong mysql-config when rebuilding 3Store.

So, why did I do all this? In a word, SPARQL. Hope it's worth it...

Donat Agosti enters the blogsphere

2006-06-19T21:55:00.000+01:00

Donat Agosti has three blogs, biosyscontext, biodivcontext, and antbase (the later is not populated yet). Not one to mince words, Donat has some pithy things to say about the E. O. Wilson Biodiversity Foundation, which looks like another example of the triumph of hype over craft (for another approach see my iSpecies).

Donat is also rather critical of the authors of a recent paper on ant phylogeny (doi:0.1126/science.1124891). Nobody can accuse ants of being dull!

Using links to rank specimens, sequences, etc.

2006-06-12T20:53:00.000+01:00

The "adundance" problem arises when a search returns too many hits. How does the user decide which ones are relevant (other than wading through the list)? The classical example is web search, where potentially millions of web pages may be returned. So, the challenge is to rank the results so the user needs look at only the top 10 or so (and is confident that what she is after is in the top 10).
For biodiversity searches this is also relevant, especially if a search may return 100s of specimens. How do we rank these (assuming some are more interesting than others)? Well, one approach is to adopt the same approach as Google -- rank things based on links. In the case of specimens, we could use links to sequences, images, and publications as evidence that a specimen "matters" (i.e., people have done work on it, and therefore it is likely to be of interest).
I'd suggested something like this for phylogenies over on iPhylo, but it dawned on me today that the same idea might make sense for the SemAnt project. As an aside, Ben Szekely (another participant at TDWG GUID2) and Elias Torres have a cool paper on extending PageRank to tags.

TDWG-GUID demo

2006-06-09T14:45:00.000+01:00

Just in time for the second meeting of TDWG GUID, I've got a version of the ant triple store up and running here. There are some technical details posted on the TDWG GUID wiki.
Note that the first time you go to the site you get a warning from Google Maps that the API key is invalid. It is, there's an issue to do with the server having multiple IPs (a temporary situation while I restore another machine that got hacked). Just click on OK and it should work fine. Figured it out. In my haste I'd left some redundant code in the file index.php, and it contained the wrong Google API key. Simple really...duh!

Discovering new things

2006-06-08T17:15:00.000+01:00

So, why bother with all this effort to aggregate information into a triple store, I hear you ask? Well, the expectation is that we can learn things we previously didn't know.
For example, consider the ant specimen casent0500379, which is recorded as the source of several sequences in GenBank. These sequences have been obtained by different research groups, and published in different papers.

We can see this immediately if we construct a graph based on the RDF in the triple store. Each node in the graph is a subject. Two nodes, x and y are connected by an edge if there is a triple corresponding to either (?x, ?pred, ?y) or (?y, ?pred, ?x). The neighbourhood of a node is all the nodes adjacent to that node.

This graph represents the neighbourhood of specimen casent0500379. I've expanded the graph by finding the neighbours of all the neighbours of casent0500379. Note that this graph shows that five sequences have been obtained from this specimen (identified by their "gi" numbers), and those five sequences are associated with three different taxonomic names in GenBank(!).

Two labs have sequenced the 28S ribosomal RNA gene from this specimen, one with accession number DQ353560 (gi:87047406), published by Moreau, et al. in Science (pmid:16601190, doi:10.1126/science.1124891) and one with accession number DQ401020 (gi:89477179), published by Ouellette et al. in MPE (pmid:16630727, doi:10.1016/j.ympev.2006.03.017).

Now, the specimen has not been identified beyond being assigned to the genus Proceratium (which includes the Google ant). These two research groups have given it different informal names, hence GenBank doesn't realise that these two taxa are the same.

In fact there is a third taxon, "Proceratium sp. CSM-2006" for this same specimen, which has been sequenced for wingless.

This is a small example of where aggregating and visualise links between multiple data sources can tell us something we didn't know before.

Postscript
My triple store misses the sixth sequence (yet another 28S rRNA sequence) from this specimen (AY325951, gi:34398469), because the specimen is recorded in the "isolate" field, not the specimen_voucher field:

     source          1..1233
                     /organism="Proceratium sp. CS-2003-1"
                     /mol_type="genomic DNA"
                     /isolate="CASENT0500379"
                     /db_xref="taxon:237763"
                     /country="Madagascar"

YeastHub

2006-06-06T15:17:00.000+01:00

YeastHub is an interesting example of data integration in bioinformatics using RDF and a triple store. See Cheung et al. for details (doi:10.1093/bioinformatics/bti1026). To quote from the abstract:

As the semantic web technology is maturing and the need for life sciences data integration over the web is growing, it is important to explore how data integration needs can be addressed by the semantic web. The main problem that we face in data integration is a lack of widely-accepted standards for expressing the syntax and semantics of the data. We address this problem by exploring the use of semantic web technologies—including resource description framework (RDF), RDF site summary (RSS), relational-database-to-RDF mapping (D2RQ) and native RDF data repository—to represent, store and query both metadata and data across life sciences datasets.

Pity the actual site seems broken...

Taxonomic Markup and GUIDs

2006-06-05T09:59:00.000+01:00

These notes were put together partly in response to discussions with Donat Agosti, but also as part of my experiments with storing ant data in a triple store. The idea is to mark up a taxonomic paper on ants with links to external sources of information (such as names, specimens, images, etc.).

If you want some related inspiration, see Leigh Dodds' post on the scientific paper as a modern palimpsest.

Firstly, I'm going to distinguish between mark up and metadata. I'm going to use mark up to mean tagging a manuscript to identify the relevant bits. For taxonomic literature this is largely after the fact, but for modern journals the article itself is represented in XML, which is then converted to a nice display using XSL. The BMC journals are a good example of this. What I'm interested in is how to mark up an article so that metadata about that article, its contents, and its relationships to other articles can be easily recovered (and output, presumably as RDF). Much of the mark up concerns the structure of a document, which in turn is important for presenting the document (say, in a web browser). I'm interested in just those bits relevant to metadata.

Identifiers
I'm assuming that we have identifiers for the items of interest (i.e., URIs such as URLs, DOIs, Handles, LSIDs). Ideally, there is a way to extract metadata about the object the identifier refers to. LSIDs provide an explicit mechanism for doing this, and CrossRef provides a service to return an XML summary of metadata held for a given DOI.
In my Taxonomic Search Engine I used the Hymenoptera Name Server's SEEK prototype to get metadata about a name, e.g. http://atbi.biosci.ohio-state.edu:210/hymenoptera/nomenclator.seek_demo?id=HNS195070 returns an XML document about the "Google ant", Proceratium google. This document contains these identifiers:

HNS153344: a taxonomic concept
HNS195070: a taxonomic name
pubHNS153344: a publication

Only the name identifier (HNS195070) has metadata that can be easily accessed in XML, as far as I can see.

How to refer to identifiers
Given the uncertainty about resolving identifiers (i.e., will LSIDs take off), one might adopt the convention used by PubMed and the BMC journals and just include the "local" part of the identifier (see example below), rather than the full blown identifier. Otherwise, a document marked up with the complete identifier will be rendered out of date if the resolution mechanism changes. In English, just use Hymenoptera Namer Server ids, not LSIDs.

Literature cited
Literature is perhaps the least problematic topic, because there are identifiers for many publications (e.g., DOIs), and tools for looking up identifiers for publications (e.g., CrossRef OpenURL, Google Scholar, PubMed, etc.).
BMC uses the following markup for a bibliograpy entry:


<bibl id="B21">
 <title>
  <p>Inter-familial relationships of the shorebirds 
  (Aves: Charadriiformes) based on nuclear DNA sequence data</p>
 </title>
 <aug>
  <au>
   <snm>Ericson</snm>
   <fnm>PGP</fnm>
  </au>
  .
  .
  .
  <au>
   <snm>Norman</snm>
   <fnm>JA</fnm>
  </au>
 </aug>
 <source>BMC Evol Biol</source>
 <pubdate>2003</pubdate>
 <volume>3</volume>
 <fpage>16</fpage>
 <xrefbib>
  <pubidlist>
   <pubid idtype="pmcid">184354</pubid>
   <pubid idtype="pmpid" link="fulltext">12875664</pubid>
   <pubid idtype="doi">10.1186/1471-2148-3-16</pubid>
  </pubidlist>
 </xrefbib>
</bibl>

Note that individual elements of the item (such as volume, pagination, etc.) are identified, but more importantly, identifiers are provided (in this case from PubMed Central, PubMed, and CrossRef). BMC is better than PLoS in this respect, as PLoS don't embed the identifiers.

Note that the mark up above embeds identifiers, not URLs (for example). URLs are fragile and can break. By just using identifiers, BMC avoids this problem, but it means that the user has to know how to make the identifier actionable.

Marking up literature to this level of detail within even a single paper would be time consuming, but as I've noted elsewhere on iSpecies, tools like ParaCite would make this tractable. ParaCite includes code to generate OpenURL requests, which means finding DOIs would be straightforward.

What's needed?
Tools to extract citations from text and locate identifiers. ParaCite would help. Relying on CrossRef's OpenURL server will be limited to those cases where CrossRef knows about the article (i.e., it has a DOI). It would be useful to have similar tools for searching PubMed, taxon-specific bibliographic databases such as FORMIS, and the Hymenoptera Name Server. By tool I mean a Web API (can be simple as an HTTP GET interface). Google Scholar would also be useful, although there are issues with using it. There is also literature in DSpace repositories, such as the AMNH's wonderful collection of their scientific publications. How do we query this on the fly? In summary, having an OpenURL interface to taxonomic literature would greatly facilitate automated mark up.

Taxonomic names
I subscribe to uBio's view (see doi:10.1080/10635150500541680) that names by themselves are useful and should be indexed. A paper may mention a name with nothing to tell us what taxonomic concept is being used. For example, this is how the Google ant paper describes the site the ants were collected from:

Exotic vegetation dominates, most notably a scrub of strawberry guava (Psidium cattleianum) and privet (Ligustrum robustrum)—but grassland and Eucalyptus plantations also occur.

The paper is about ants, and for Discothyrea berlita Fisher, Proceratium avium Brown, Proceratium avioide de Andrade, and Proceratium google Fisher, we have a clear concept of what those names refer to (at a minimum, the specimens listed). For the other names (which include ants and plants), we have little to go on apart from the names. So, every occurrence of a taxonomic name in the document should be flagged. BioOne journals do this already. Any scientific name in the HTML is linked to ITIS. The linking is not intelligent as it is a search, not a link to an identifier (i.e., nobody actually checks that ITIS has the name).

What's needed?
Names in the document should be linked to uBio namebank LSIDs. For ants we could also use the Hymenoptera Name Server.

Taxonomic concepts
If we think of a concept as "what the name means", then this is most relevant to taxonomic papers describing names (e.g., where the author lists specimens, describes features of the taxon), argues that two taxa are synonyms, etc. uBio has a notion of a concept in the sense that a name may exist in multiple classifications, and each combination of name and classification has its own identifier. Hymenoptera has concepts (my understanding is that this is what one sees when one searches the Name Server through the web interface).

Treatment
The taxonomic treatment is probably the same as the taxonomic concept (or perhaps, a treatment can be regarded as a detailed taxonomic concept). In my early experiments I assigned GUIDs to the taxonomic treatment, and used the Dublin Core tag <dcterms:isPartOf> to associate each treatment with the publication. As a quick hack the identifier for each treatment within a paper included the XPath query that would locate that treatment in the larger document, e.g. //tax:treatment[1]. Embedding this much meaning in an identifier is probably not wise, but it meant that given just the identifier, one could extract the treatment from the document.

Authors
The lack of GUIDs for authors is a long standing issue.

Images
Most taxonomic images probably reside solely within the publication, but some may be stored in external databases (such as AntWeb). In the later case, the mark up should make the link to the external source.

Specimens
If the specimen has an electronic existence, then link to that.

Truncating strings using XSL

2006-05-31T18:37:00.000+01:00

Problem
I want to display a RSS feed from Connotea for papers I've tagged with "Formicidae". The titles of these papers can be long, e.g.:

Dracula ant phylogeny as inferred by nuclear 28S rDNA sequences and implications for ant systematics (Hymenoptera: Formicidae: Amblyoponinae)

This takes up too much space when displayed in a panel on the web page, so I want to truncate the titles in a sensible way.

Solution
aaronland.info has a nice solution that truncates to a fixed length, optionally at a word boundary. Our verbose title becomes:

Dracula ant phylogeny as inferred by nuclear 28S...

Much nicer.

(Via Google: xsl truncate string.)

Currently playing in iTunes: Dani California by Red Hot Chili Peppers

Australian ants online

2006-05-28T23:08:00.000+01:00

The Australia National Insect Collection has an online resource of Australian ants that provides species maps and Google Earth files of specimen distributions.

A potentially useful source of data, but the KML files don't include specimen codes, and there is an unfortunate disconnect between specimen codes and ids used to link to them. For example, in the specimen list for Myrmecia midas, the first specimen has a "MaterialID" of 128315, yet the ANIC accession code for this specimen is 32-012805. Pity this identifier is not made of in the link to the data.

The site also displays specimen metadata as an HTML table -- this information is also available as a CSV file. This is an example where a touch more effort would make it truly useful as a source of data. Something as simple as RDF with the specimen URL as URI would be a big help, but this could be achieved by some judicious screen scraping...

Duh!
Just realised, the data is served up by GBIF, which makes life much easier.

SemAnt?

2006-05-09T18:58:00.000+01:00

Semantic Ants (geddit?).

Initially I blogged this project on iPhylo, such as an introduction to populating the triple store, and thoughts on how to automatically add to its contents.

As the project develops I hope to describe further data sources, possible queries, user interface design, and benchmarks.