In my work with BBC Journalism we have been attempting to take the philosophy of Tom Scott’s Wildlife Finder and applying it to News and Sport. Our starting point has been the Winter Olympics.

The step change was in creating a populated domain model for the games. The things that made up this vocabulary were used by journalists to tag their stories. The tagged stories were then aggregated automatically onto sports indexes. This allowed us to create many more indexes than would have been possible with manual management.

Overall the project was a great success but it raised some interesting questions. The design of the indexes was created by the user experience team. The algorithms were written by developers and informed the ordering of the stories.  This left journalists to simply tag stories and watch their stories appear on indexes they had no control over. It certainly felt like their influence on part of the product had moved a step away from them. This was reflected in journalists’ feedback and the frequent questions about how to game the system to control the order of stories on indexes.

So the questions are:

• How do you enable the journalists to feel in control of the story telling?
• How to do this without introducing tags for value judgements?
• How do you ensure that the site has voice and feels editorialised – as opposed to being simply lists of dynamically aggregated data?

Tom Scott has convinced me the answer is the concept of the collection (and variations on this theme). The collection replicates the manually managed index of stories with a structured list of things. The Wildlife finder example is David Attenborough’s favourite moments. A very simple example for sport might be the best goals of the World Cup. Although this does not seem particularly radical, the beauty of it is that the curatorial layer is built on top of a domain modelled approach.

Because the things that live in our model are associated with assets and data,  the journalist, in selecting a thing to include in a collection pulls data through the system.

Take the same example of the best goals of the World Cup. A journalist would select their top ten goals of the tournament. As the journalist identifies and pulls things through the system into the collection the context around those goals are pulled with them. So the game they were scored in, the importance it had and information about the goal scorers record in the tournament.

Why it is not tagging:

It is important to distinguish the process of creating a collection from the act of tagging. Tagging associates content with things in the domain model. Journalists tagging stories ensure we build up a consistent mapping of the editorial content to the things (and/or concepts) in our domain.

The process of creating collections is closely tied to the editorial judgement of those curating them. Tagging clips with the tag good goal and then anonymously aggregating them is not.

Why it empowers journalists:

The Guardian has found the balance in their topic pages by allowing an editor to pick a story to be displayed at the top of every automated page. But does this go far enough? This still sits very much within the document model of storytelling. What a collection (or similar) begins to allow is a true web adaptation of a news story.

It is the curatorial layer and the use of collections that will allow organisations to reflect voice, perspective and expertise.  How this will improve the experience for the news reader will be the subject of this blog over the forthcoming months.

Could the means by which news organisations adapt their story telling using tools like collections be the key to their ongoing survival?

A topic that interests me is the relationship between Linked Data and controlled vocabularies, to steal a phrase from Tom Coates (native to the web), and Linked Data’s call for vocabularies native to the web.

Let’s look at it this way – if you were asked to creating a web presence for an individual or organisation today you might propose the following:

1. Make interesting documents public.
2. Publish using web standards such as HTML.
3. Provide useful information about the individual or organisation.
4. Link to similar documents where you can.
5. Then if the documents are useful and you are gracious in linking to others they will link back to you.

It is apparent that Linked Data asks the same of controlled vocabularies.

1. Make your vocabularies public.
2. Publish using the web standards of Linked Data.
3. For each concept provide useful information for humans and machines.
4. Link to other vocabularies (map concepts) where you can.
5. If you have provided a useful set of concepts and relationships others will link back to you, increasing the value of your CV.

It could seem crazy at the moment to give away your taxonomy for free but it would have been a similarly difficult argument to convincing an organisation to have a web presence ten or fifteen years ago.

Linked Data is already showing the benefits of this approach. When we open-source vocabularies we can be much more ambitious in the richness of relationships and complexity of structures.  In my talk I mentioned that the, wonderful, Wildlife Finder would not have been feasible had the ontologies not been publically available to use and build upon.  A Wildlife Finder built on a far simpler BBC bespoke taxonomy of animals, habitats and behaviours would have been a far poorer and more costly proposition. Martin expands on this in his Guardian post.

Recently we have seen the likes of LCSH and New York Times vocabularies joining the Linked Data cloud and becoming web native vocabularies. I suspect the success and survival of many vocabularies will depend on how quickly their owners can grasp the importance of becoming open and native to the web.

Government data panel - photo by Zac Beauvais

Having just recovered from last week’s London Linked data meet up.  I thought it was time to collect together the talks and commentary from the day.

Georgi and I are particularly grateful to everyone for coming, in particular those that spoke.  A special thank you also to  Talis for picking up the bar tab.

I think Zach Beauvais summarised the day nicely in his post:

‘The day was a storming success, with talks and presentations from all over the Linked Data community: from academia to startups. I think the organisers were slightly overwhelmed, because in the end there were nearly 200 people there, making use of the Talis-sponsored bar well into the evening. Apart from being a good opportunity to catch up with people, this meetup had the feeling of a guild-meet of Linked Data professionals—with lots of different perspectives over similar problems.’

Presentations

Here are links to the presentations so far and I will add the rest as they become avaliable:

Tom Scott / Yves Raimond (BBC)

“Contextualising BBC programmes using the Data Web”

The BBC, following the Linked Data principles, now publishes a URI for every TV and Radio programme it broadcasts this allows people to browser by schedule, genre, format and a-z.

More recently we have published URIs for music artists, animal species and habitats – these pages not only provide useful information in their own right but also allow us to re-contextualise the programme information helping users to discover new content and new patterns.

Leigh Dodds (Talis)

“DataIncubator.org — What Is It & What’s In It?“

This talk will introduce the dataincubator.org project which, supported by the Talis Connected Commons scheme, provides an umbrella project for publishing public domain linked data, with the aim of demonstrating to the original publishers the benefits of Linked Data, as well as a means to build on the community’s efforts. The talk will review the project and some of the datasets that have currently been made available.

Andrew Walkingshaw (Timetric)

“Time to build: storing, sharing and analysing statistics with Timetric, a Web-native service for managing numbers”

Timetric is a Web service which lets users upload, download, visualize and set up calculations on over a hundred thousand different measurements, the values of all of which are tracked over time. But how would you build that, and when you have, who’d want it? In this talk, we’ll discuss the lessons we’ve learned in building a service for sharing open data on the Web and in building a business around that service.

Michael Smethurst, Matthew Wood (BBC)

“Rights, Privacy and Linked Data“

Georgi Kobilarov (Freie Universität Berlin / DBpedia)

“Integrating Linked Data”

Nigel Shadbolt (University of Southampton)

“Hard Research Challenges in the Web Of Linked Data: The EPSRC EnAKTinG Project”

The UK Engineering and Physical Sciences Research Council (EPSRC) has funded a three year two million pound project at the University of Southampton to investigate the challenges represented by the Web of Linked Data. Nigel Shadbolt and Tim Berners-Lee are two of the Principal Investigators on this project. In this brief presentation the projects aims and ambition will be outlined – together with progress to date.

Libby Miller (BBC / NoTube)

“Beancounter – telling you about you”

Increasing automation means that lots of data is available about what you do, including what you watch and listen to. This means that companies or researchers can mine information about your activities and use them to make predictions about what you might like, and what they might be able to sell you. Beancounter uses attention data from multiple sources, enhanced by linked data, to tell you what you are *really* interested in – rather than what you *think* you are interested in. It puts the control about what sources can be mined in your hands, and limits what companies can do with the outputs. Beancounter is a product of the NoTube EU project.’

Richard Cyganiak (DERI Galway)

“Sig.ma – Live Views on the Web of Data”

Increasing amounts of high-quality data are being published on the web of data, but a lack of applications for searching and browsing it makes access and exploration difficult. Sig.ma is a new user interface that improves upon previous ones by offering fine-grained control over source selection, fuzzy entity matching, and schema and value consolidation. Sig.ma is online at http://sig.ma/… and provides the fastest way yet to get an overview about the data available on a given topic.

Jun Zhao (University Oxford)

“Linked Data for Connecting Medicine Knowledge”

Mischa Tuffield / Steve Harris (Garlik)

“Making FOAF useful: http://foaf.qdos.com/ “

Since the beginning of the Linked Data Movement, a fair chunk of the resolvable RDF found on the web has been FOAF data. This talk will involve a brief overview of what FOAF represents, a list of the services we provide, how we go about saving public and private FOAF data, whilst presenting insight into the technologies used to underpin the services on foaf.qdos.com.

Ian Millard (University of Southampton)

“RKB-Explorer”

The RKBExplorer.com application provides a simple interface over multiple Linked Data sources to assist with the discovery and exploration of related activities with the academic research domain.

This talk will briefly summarise issues and experiences regarding interoperation of multiple sources, and outline some of the services we offer that can be used by all.

Panel: Government Data

Chair: Carol Tullo (Office of Public Sector Information)
Tim Bernes-Lee
Paul Miller (Cloud of Data)
Nigel Shadbolt (University of Southampton)
Mark Birbeck (webBackplane)
John Goodwin (Ordnance Survey)

A good summary of this panel in a blog post by Jane Stevenson and in Zac’s post.

‘It gave a good sense of what is happening at the moment with Linked Data and what the issues are. Tim Berners-Lee (inventor of the Web) and Nigel Shadbolt talked about the decision to prioritise UK government data within the Linked Data project – clearly it is of great value for a whole host of reasons, and a critical mass of data can be achieved if the government are on board, and also we should not forget that it is ‘our data’ so it should be opened up to us – public sector data touches all of us, businesses, institutions, individuals, groups, processes, etc.’

Thank you to Carol Tullo for doing such a good job of chairing the session.

Panel: Future of Journalism

Chair: Paul Bradshaw (Online Journalism)
Martin Belham (The Guardian)
John O’ Donovan (BBC)
Dan Brickley
Leigh Dodds (Talis)

A number of posts are avaliable from this session from the wonderful chair Paul Bradshaw and panelist Martin Belham of the Guardian.

Data and the future of journalism panal discussion: Paul Bradshaw

Linked data future of journalsim part 1: Martin Belham

Linked data future of journalsim part 2: Martin Belham

Tom Heath’s “Linked Data – The Story So Far” was a fantastic way to finish the evening and really captured the challenges that lie ahead.

Update: Two further post from Martin Belham: Part 1 and Part 2.

Some of the largest and most successful websites have taken simple narrative structures and made them scale successfully.  In the mold of the fairytale “once upon a time” and “they all lived happily ever after” these sites have come to own their simple narrative structures and this has played a significant part in their success.  Some familiar examples:

• Customers Who Bought This Item Also Bought – noun (book) verb (also bought) noun (book)
• Buy it now – noun (user) verb (buy) noun (item)
• Such and such wrote on your Wall – noun (friend) verb (wrote on) noun (wall)

These simple noun-verb-noun narratives should be familiar and are very much part of the brand of these sites. This is a result of them getting these narratives to scale and ensuring there is the quality of data to back them up.

Now in order to make sure these narratives are applied consistently as the site accumilates content these structures need to be understood by your application. This means the noun-verb-noun structures must be encoded into your domain model ( and so your database) from the outset. Designing the site in this way means that as new content, pages and data are added to the site these narrative structures will be automatically created. This guarantees new pages are incorporated into the site and automatically become a scene in the sites larger story.

Weak and strong narrative structures

As we move from flat published pages to large dynamically created sites we need to think more and more about the primary narrative structures. These user journeys will be encoded into the very core of the site and you will want to be confident you have selected the right ones and that there is the data to back them up.

One of the strengths of the BBC News site is its contextual navigation with strong narrative. For example a BBC News story about Kosovo will carry an explicit user journey to the background story of the independence of Kosovo. This is in contrast to tags. Tags help to open up new user journeys but are weak in narrative, taking the form ‘this content is about this tag’. Related links also often fall into this category of weak narrative. One of the problems with rich narrative structure is that they are difficult to scale, this poses a significant challenge.

Web-scale narratives

When George Lucas was looking for a narrative structure for the beginning of his Star Wars films he used a well understood simple narrative structure, ‘once upon a time’.

He knew that this would be something that his audience would immediately understand.

The dream of the Semantic Web project follows a similar logic. Take the simple narrative structures that have been so successful in creating user journeys within large scalable websites and apply them to the web at large.  This means narratives (in the form of domain models and ontologies) that are not limited to a single site. Not just ‘people who bought this on Amazon also bought this’ rather ‘people who bought this on the web also bought this’ web-scale narrative structures. This will not only help create more coherent user journeys across the web but also provides more structure to help machine understanding.

“Unless anyone here works for Philips, I’m fairly certain that nobody in this room wants or will ever buy a networked fridge.”

http://www.currybet.net/cbet_blog/2008/09/euroia2008_part1.php

Fair point but I wanted to revisit the concept with regard to the big challenge of this century; climate change and energy conservation. Thomas Friedman’s book Hot, Flat and Crowded is a nice summary of some of the issues and possible solutions.

The problem:
If we continue on our current path CO2 levels will double (to 560ppm) around the mid-century and will triple by 2075. A situation we have not been in for 650,000 years. We don’t know what it will be like to live in a 560ppm CO2 world let alone an 800ppm one.

“So now we have a target: We want to avoid the doubling of CO2 by mid-century, to do it we need to avoid emission of 200 billion tons of carbon as we grow between now and then.”

Thomas Friedman

Solutions:
Freidman identifies a number of targets that need to be met. One of them is to cut electricity use in homes, offices, and stores by 25%. A way that this might be achieved according to Friedman is to become more intelligent about energy use and the development of an Energy Internet. Energy distribution and consumption is currently stuck in the 50’s and has failed to embrace the IT revolution.

The concept of an Energy Internet was originally conceived in an Economist article:

“Energy visionaries imagine a “self-healing” grid with real-time sensors and “plug and play” software that can allow scattered generators or energy-storage devices to attach to it. In other words, an energy internet.”

http://www.economist.com/science/tq/displaystory.cfm?story_id=E1_NQSGJRR

Amongst other things this would mean more intelligent appliances in the home that can negotiate their energy needs with the grid as well as communicating to the homeowner the worst offenders in growing energy bills. Friedman imagines what it might be like to live with a smart grid.

“..an Energy Internet in which every device – from light switches to air conditioners, to basement boilers, to car batteries and power lines and power stations – incoporate microchips that could inform your utility of the energy level at which it was operating, take instructions from you or your utility as to when it should operate and at what level of power, and tell your utility when it wanted to purchase or sell electricity. You and your utility now have two-way communications.”

Thomas Freidman

So the smart fridge is not dead but it just won’t be doing the weekly shop for us, it will be helping save the planet (or at least your energy bill).

Clearly ubiquitous computing is closely tied to the Semantic Web. Until machines can parse the web on our behalf we are stuck with large screens so that we can parse the data for them. The smart fridge will need Semantic Web technologies and so link into a larger body of data about our energy use; where it comes from (clean or dirty), how it is being used in the home and the damage it is doing. Targeted advertising will illustrate how new, more efficient, appliances will impact our energy use and so on into the graph…

Currently we have little context to understand our energy use and context is king when it comes to education and driving real changes in behaviour. Perhaps the Energy Internet and the tackling of one of the big problems will be the making of the Semantic Web project.

Update: The Talis people have written a nice post about semweb and the home: http://blogs.talis.com/nodalities/2008/12/smart-stuff.php

Chris covers most of the issues we discussed in his post. But one thing we did not cover is the interesting way Wikipedia handles categories. I have tried to find discussion about this approach with no success. It will be an interesting issue to raise in the ISKO mail group.

As opposed to making one entry a parent of another, as we might do in a taxonomy. In Wikipedia categories (groupings of concepts) are treated as a completely different type of entry. This means we can have groups like Dog-related_professions_and_professionals without falling into the trap of treating them like concepts (entries) in their own right.

The two activities of defining a concept and grouping associated concepts are separated. The result of this is Wikipedia entries remain topics of interest that are discrete and clearly defined. This make them ideal for sanity checking that our choice of topic aggregation page are discrete and clearly defined as discussed in the previous ‘what makes a good topic aggregation page?’ Post.

Update: Bob Bater has written a nice post on  this topic on the ISKO UK blog.

Conclusion? The Wikipedia categorization system reflects but does not consistently apply the principles of KO as expounded in the formal literature. It is nevertheless interesting because it might well represent what results when folksonomy meets formal KO and agrees to a compromise.

http://iskouk.wordpress.com/2008/09/22/wikipedias-approach-to-categorization/

In order to fulfill these goals I have been thinking about what makes a good topic for aggregation? To answer this question I went back to have a look at Donna Maurer’s discussion of basic level categories. Most of the introduction below comes from her presentation.

What are basic level categories?

How to spot a basic level category:

• A basic level category is somewhere in the middle of a hierarchy and is cognitively basic
• It is the level that is learned earliest
• Usually has a short name and is used frequently
• Highest level at which a single mental image can reflect the category
• There is no definitive basic level for a hierarchy it is dependent on the audience

Why are basic level categories relevant to Topic Pages?

Basic level categories have some characteristics that make them of interest to Topic Pages:

• Things are remembered more readily at basic level.
• People name things more readily at basic level.
• Languages have simpler names at basic level.

Donna explains…

In short, people naturally, at a deep cognitive level, deal easier with basic level categories.

It is important to understand that basic level categories are not just easier on a superficial level, because they are shorter or something. Cognitive scientists say that basic level categories are cognitively real. They seem to be ingrained in the human mind somehow, in a way that makes it easier for us to deal with basic level categories.

The fact that people name things more readily at basic level could lead us to hypothesize two things:

• When constructing a keyword search users are more likely to use basic level categories.
• Users are more likely to visit links labeled with basic level categories.

If this is the case then when selecting candidate Topic Pages we should look to select ones that sit at a basic level.

Basic level categories are good for SEO

Topic Pages are specifically designed to attract traffic from search engine queries and drive users to BBC content, ideally ranking highly for certain keywords in external engines. So they need to match users’ keywords as much as possible. The process of matching search keywords to pages is well documented in SEO and called keyword optimisation.
Selecting what level of granularity to create Topic Pages at is something we have been exploring in the range of Topics selected for the beta launch. A little research should help us identify the keywords that sit at the basic level of granularity. I attempted to do this by looking at the average UK keyword searches for a series of topics.

Taking a range of related concepts from different levels of a hierachy we might expect to find as we move up the tree the more we find people searching with the term, because it is more general and incorporates more concepts. The theory of basic level categories would challenge this and expect the middle basic level concept to be the most used.

Look at this example taken from Google UK on the 17 July 2008.

Here the keyword dogs holds the middle ground between abstraction and detail. I think this example illustrates that search logs could be used to assist in the identification of basic level categories.

Of course this is not always the case

In this case the more abstract keyword is far more popular. This does not necessarily make it a better candidate for a Topic Page, for example it might be a difficult Topic to train due it being difficult to define.
As a rule popular search keywords indicate suitable Topics and in the majority of cases the more popular search keywords will sit at a basic level.
In the instances where super-categories are more popular consideration must be given to the extra problems this might cause the Topic Page, for example editorial overhead and awkward user navigation of the Topic as a link label.

Basic level categories are good for navigation

In addition to being good SEO practice I would also suggest that selecting basic-level categorises could improve user navigation. An example might be on contextual links where we present the name of a Topic as the link label. This assumption is based on information foraging theory.

Jared Spool has applied information foraging theory to web design with what he calls the “scent of information.” In order to efficiently and effectively forage for information on the web, seekers need to have a sense of where they are going. The design of a navigation system should provide users with an accurate “scent” that they can follow to their destination.

We have argued that basic level categories are more easily grasped than higher level categories. An example might be the categories cars (basic) as compared to vehicles (super). With the category vehicles you have a less clear sense, scent or image of what is included in this group. This could result in a lower click through rate of users from links to Topics.

Basic level catergories are good for editorial staff

Having monitored a number of Topic Pages for the last couple of months we are beginning to get a sense of what makes an easy to manage topic.

An easy to maintain Topic is:
• specific
• easily defined
• discrete

An example of an awkward concept would be crime. A Topic like this makes it hard for editorial staff to judge whether a piece of content should or should not be included. This is because it contains a list of concepts that will vary depending on who you ask and this list is liable to change over time. In the same way we talk about conceptually awkward categories for users a similar problem is presented to the editorial staff maintaining the pages.

This is apparent if you think about selecting a set of representative documents for training a Topic Page about dogs as opposed to a Topic Page about crime. As a Topic dogs is easy to train and easy to review. Content is fairly clearly about dogs or not. This is again an example of the benefits of selecting basic-level categories.

Conclusion

On reflection Topics like elected assemblies do not look like such a good idea. The individual elected assemblies as Topics would have made much better pages.

Not only are they unlikely to be searched for but they are also confusing to users when shown as contextual links. In addition they are difficult to maintain for editorial staff, again because of their conceptual awkwardness.

The following recomendations should be considered when selecting a suitable topic for an aggregation page.

Pick Topics that sit at a basic level between detail and abstraction. These concepts can be identified through the following criteria:

• frequently used in keyword searches
• represent a discrete concept
• are easily defined

It will be possible to say: In language we have different kinds of word. For the functions of the word “slab” and the word “block” are more alike than those of “slab” and “d”. But how we group words into kinds will depend on the aim of the classification, and on our own inclination.

The types of associations and the members included in a group will depend on the ‘aim’ of this grouping. The classical model would suggest that we discover categories; they are in some way natural, necessary and objective. Wittgenstein challenges this model and encourages us to look at the context of the language use to understand the meaning of a categorisation.

For me this reinforces the point that any categorisation scheme is only really meaningful with consideration of its intended use and this is essentially what this blog is about.