We express our identities through our collections. Online these collections take the form of Amazon wishlists, Last fm playlists and lists of friends on Facebook. Perhaps less consciously we have search histories, purchase profiles and a trail of cookies picked up from website visits.

In David Siegel’s book Pull he posits a future where our personal details are consolidated in a private space in the web:

Your personal data locker will store your personal ontology. It helps you find television shows and movies, it helps you learn about wines you might enjoy, it helps you find bargins online, plan a trip, find events you might want to attend, or spot a new restaurant, and it can help with dating life if you’re single. Hook it in to your everyday activities and you’ll build an ontology with millions of triples, all of which make your data locker into a ’smart’ virtual assistant that continues to learn as you go through the day.

Few news organisations have attempted to bridged this gap between the news story and our personal profiles.  The New York Times perhaps being the exception taking a users LinkedIn account, looking at your area of work and then serving contextual stories and ads related to your area of work.

In some respects SEO (and the optimising of keywords in story titles) could be considered a crude attempt by news organisations at mapping stories to the profiles (keyword search patterns) of their intended audience. We have recently seen a move away from SEO effort in the news industry in favour of building more meaningful relationships with loyal customers.

I suspect with time we will see a focus of effort on mapping the model of the news domain to the domain of the user (personal data locker). Relating the context of the story to the things of importance in our world; the topics, events, work, people and hobbies.

How does moving from a document focused view of the world to a thing focused view change the role of the collection?

We took Wildlife Finder as our example. Wildlife Finder is built upon a domain modelled approach and dynamically aggregates content and data around the ‘things’ in the model. Collections can then be used to build editorial layers on top.  As Tom Scott points out:

Collections allow us to curate a set of resources – to group and sequence clips and other resources to tell stories like the plight of the tiger or the years work of the BBC’s natural history unit.

Tom goes on to say that by releasing the data for Wildlife Finder it means that “our audiences and ‘users’ could also build stories”.

Perhaps the most striking example of how the user creation of collections can be used to tell stories is by the use of data filtering tools such as Parallax and Microsoft Pivot.

In Pivot’s own words:

In short, datasets are organized as collections. Results can be as granular or as big-picture as the user desires, and correlations and patterns are easy to see and examine through powerful but simple visualizations. Imagine browsing through thumbnails representing Kiva loans, then sorting the loans by the different types of businesses they helped established.

In order for Pivot to work datasets need to be in a certain format. I suspect that Linked Data will lend itself to these types of tools and products like Wildlife Finder that have focused on curating context as opposed to curating content will benefit greatly.

Collections part 3: You as a collection

Link journalism is linking to other reporting on the web to enhance, complement, source, or add more context to a journalist’s original reporting.

Scott Karp

How can these collections of links be best used to serve the core principles of journalism?

The BBC’s use of external links to cite sources has been criticised for not linking to the original source of a story.

Paul Bradshaw has written an excellent post on the subject and makes the following point:

In an online environment one of the biggest signals in how we build a picture of the trustworthiness of someone or something is the links surrounding it. Who is that person friends with? What does this website link to? Who gathers here? What do they say? What else does this person do? What is their background, their interests, their beliefs?

While we increasingly talk about the role of publishers as curators of content [caveat], we should perhaps start thinking about how publishers are also curators of context.”.

I find the distinction between the curation of content and the curation of context very useful.  Paul highlights the value of using links to place the story in its context as opposed to merely pointing to similar content about the same story.  In addition it also puts the source referenced by the BBC into context by saying something about how the BBC regards it.

BBC Journalism currently use several quite different strategies for linking to both related BBC stories and other sites on the web. The most common are the ’see alsos’ and the ‘related internet links’ that appear on stories.

These links are picked by the journalist as related in some way to the story.  Generally the links sit in a template that is reused for similar stories so they tend to be fairly non-specific, often linking to home pages of sites rather than deep links to sources.  They typically perform poorly in terms of click-throughs.

A different strategy is illustrated by the BBC Sport football gossip column. The column is created daily and provides an overview of the day’s football gossip. Short summaries of stories are written and then published with a link to the full story in the original source.

In comparison to the related internet links on story pages a significant amount of BBC Journalism’s external referrals go through this one page. The gossip column is a testament to how external links can be used in a meaningful and useful way.

Collections part 2: Collections of things

In my work with BBC Journalism we have been attempting to take the philosophy of Tom Scott’s Wildlife Finder and applying it to News and Sport. Our starting point has been the Winter Olympics.

The step change was in creating a populated domain model for the games. The things that made up this vocabulary were used by journalists to tag their stories. The tagged stories were then aggregated automatically onto sports indexes. This allowed us to create many more indexes than would have been possible with manual management.

Overall the project was a great success but it raised some interesting questions. The design of the indexes was created by the user experience team. The algorithms were written by developers and informed the ordering of the stories.  This left journalists to simply tag stories and watch their stories appear on indexes they had no control over. It certainly felt like their influence on part of the product had moved a step away from them. This was reflected in journalists’ feedback and the frequent questions about how to game the system to control the order of stories on indexes.

So the questions are:

• How do you enable the journalists to feel in control of the story telling?
• How to do this without introducing tags for value judgements?
• How do you ensure that the site has voice and feels editorialised – as opposed to being simply lists of dynamically aggregated data?

Tom Scott has convinced me the answer is the concept of the collection (and variations on this theme). The collection replicates the manually managed index of stories with a structured list of things. The Wildlife finder example is David Attenborough’s favourite moments. A very simple example for sport might be the best goals of the World Cup. Although this does not seem particularly radical, the beauty of it is that the curatorial layer is built on top of a domain modelled approach.

Because the things that live in our model are associated with assets and data,  the journalist, in selecting a thing to include in a collection pulls data through the system.

Take the same example of the best goals of the World Cup. A journalist would select their top ten goals of the tournament. As the journalist identifies and pulls things through the system into the collection the context around those goals are pulled with them. So the game they were scored in, the importance it had and information about the goal scorers record in the tournament.

Why it is not tagging:

It is important to distinguish the process of creating a collection from the act of tagging. Tagging associates content with things in the domain model. Journalists tagging stories ensure we build up a consistent mapping of the editorial content to the things (and/or concepts) in our domain.

The process of creating collections is closely tied to the editorial judgement of those curating them. Tagging clips with the tag good goal and then anonymously aggregating them is not.

Why it empowers journalists:

The Guardian has found the balance in their topic pages by allowing an editor to pick a story to be displayed at the top of every automated page. But does this go far enough? This still sits very much within the document model of storytelling. What a collection (or similar) begins to allow is a true web adaptation of a news story.

It is the curatorial layer and the use of collections that will allow organisations to reflect voice, perspective and expertise.  How this will improve the experience for the news reader will be the subject of this blog over the forthcoming months.

Could the means by which news organisations adapt their story telling using tools like collections be the key to their ongoing survival?

Collections part 1: Collections of links

A topic that interests me is the relationship between Linked Data and controlled vocabularies, to steal a phrase from Tom Coates (native to the web), and Linked Data’s call for vocabularies native to the web.

Let’s look at it this way – if you were asked to creating a web presence for an individual or organisation today you might propose the following:

1. Make interesting documents public.
2. Publish using web standards such as HTML.
3. Provide useful information about the individual or organisation.
4. Link to similar documents where you can.
5. Then if the documents are useful and you are gracious in linking to others they will link back to you.

It is apparent that Linked Data asks the same of controlled vocabularies.

1. Make your vocabularies public.
2. Publish using the web standards of Linked Data.
3. For each concept provide useful information for humans and machines.
4. Link to other vocabularies (map concepts) where you can.
5. If you have provided a useful set of concepts and relationships others will link back to you, increasing the value of your CV.

It could seem crazy at the moment to give away your taxonomy for free but it would have been a similarly difficult argument to convincing an organisation to have a web presence ten or fifteen years ago.

Linked Data is already showing the benefits of this approach. When we open-source vocabularies we can be much more ambitious in the richness of relationships and complexity of structures.  In my talk I mentioned that the, wonderful, Wildlife Finder would not have been feasible had the ontologies not been publically available to use and build upon.  A Wildlife Finder built on a far simpler BBC bespoke taxonomy of animals, habitats and behaviours would have been a far poorer and more costly proposition. Martin expands on this in his Guardian post.

Recently we have seen the likes of LCSH and New York Times vocabularies joining the Linked Data cloud and becoming web native vocabularies. I suspect the success and survival of many vocabularies will depend on how quickly their owners can grasp the importance of becoming open and native to the web.

Some of the largest and most successful websites have taken simple narrative structures and made them scale successfully.  In the mold of the fairytale “once upon a time” and “they all lived happily ever after” these sites have come to own their simple narrative structures and this has played a significant part in their success.  Some familiar examples:

• Customers Who Bought This Item Also Bought – noun (book) verb (also bought) noun (book)
• Buy it now – noun (user) verb (buy) noun (item)
• Such and such wrote on your Wall – noun (friend) verb (wrote on) noun (wall)

These simple noun-verb-noun narratives should be familiar and are very much part of the brand of these sites. This is a result of them getting these narratives to scale and ensuring there is the quality of data to back them up.

Now in order to make sure these narratives are applied consistently as the site accumilates content these structures need to be understood by your application. This means the noun-verb-noun structures must be encoded into your domain model ( and so your database) from the outset. Designing the site in this way means that as new content, pages and data are added to the site these narrative structures will be automatically created. This guarantees new pages are incorporated into the site and automatically become a scene in the sites larger story.

Weak and strong narrative structures

As we move from flat published pages to large dynamically created sites we need to think more and more about the primary narrative structures. These user journeys will be encoded into the very core of the site and you will want to be confident you have selected the right ones and that there is the data to back them up.

One of the strengths of the BBC News site is its contextual navigation with strong narrative. For example a BBC News story about Kosovo will carry an explicit user journey to the background story of the independence of Kosovo. This is in contrast to tags. Tags help to open up new user journeys but are weak in narrative, taking the form ‘this content is about this tag’. Related links also often fall into this category of weak narrative. One of the problems with rich narrative structure is that they are difficult to scale, this poses a significant challenge.

Web-scale narratives

When George Lucas was looking for a narrative structure for the beginning of his Star Wars films he used a well understood simple narrative structure, ‘once upon a time’.

He knew that this would be something that his audience would immediately understand.

The dream of the Semantic Web project follows a similar logic. Take the simple narrative structures that have been so successful in creating user journeys within large scalable websites and apply them to the web at large.  This means narratives (in the form of domain models and ontologies) that are not limited to a single site. Not just ‘people who bought this on Amazon also bought this’ rather ‘people who bought this on the web also bought this’ web-scale narrative structures. This will not only help create more coherent user journeys across the web but also provides more structure to help machine understanding.

Chris covers most of the issues we discussed in his post. But one thing we did not cover is the interesting way Wikipedia handles categories. I have tried to find discussion about this approach with no success. It will be an interesting issue to raise in the ISKO mail group.

As opposed to making one entry a parent of another, as we might do in a taxonomy. In Wikipedia categories (groupings of concepts) are treated as a completely different type of entry. This means we can have groups like Dog-related_professions_and_professionals without falling into the trap of treating them like concepts (entries) in their own right.

The two activities of defining a concept and grouping associated concepts are separated. The result of this is Wikipedia entries remain topics of interest that are discrete and clearly defined. This make them ideal for sanity checking that our choice of topic aggregation page are discrete and clearly defined as discussed in the previous ‘what makes a good topic aggregation page?’ Post.

Update: Bob Bater has written a nice post on  this topic on the ISKO UK blog.

Conclusion? The Wikipedia categorization system reflects but does not consistently apply the principles of KO as expounded in the formal literature. It is nevertheless interesting because it might well represent what results when folksonomy meets formal KO and agrees to a compromise.

http://iskouk.wordpress.com/2008/09/22/wikipedias-approach-to-categorization/

In order to fulfill these goals I have been thinking about what makes a good topic for aggregation? To answer this question I went back to have a look at Donna Maurer’s discussion of basic level categories. Most of the introduction below comes from her presentation.

What are basic level categories?

How to spot a basic level category:

• A basic level category is somewhere in the middle of a hierarchy and is cognitively basic
• It is the level that is learned earliest
• Usually has a short name and is used frequently
• Highest level at which a single mental image can reflect the category
• There is no definitive basic level for a hierarchy it is dependent on the audience

Why are basic level categories relevant to Topic Pages?

Basic level categories have some characteristics that make them of interest to Topic Pages:

• Things are remembered more readily at basic level.
• People name things more readily at basic level.
• Languages have simpler names at basic level.

Donna explains…

In short, people naturally, at a deep cognitive level, deal easier with basic level categories.

It is important to understand that basic level categories are not just easier on a superficial level, because they are shorter or something. Cognitive scientists say that basic level categories are cognitively real. They seem to be ingrained in the human mind somehow, in a way that makes it easier for us to deal with basic level categories.

The fact that people name things more readily at basic level could lead us to hypothesize two things:

• When constructing a keyword search users are more likely to use basic level categories.
• Users are more likely to visit links labeled with basic level categories.

If this is the case then when selecting candidate Topic Pages we should look to select ones that sit at a basic level.

Basic level categories are good for SEO

Topic Pages are specifically designed to attract traffic from search engine queries and drive users to BBC content, ideally ranking highly for certain keywords in external engines. So they need to match users’ keywords as much as possible. The process of matching search keywords to pages is well documented in SEO and called keyword optimisation.
Selecting what level of granularity to create Topic Pages at is something we have been exploring in the range of Topics selected for the beta launch. A little research should help us identify the keywords that sit at the basic level of granularity. I attempted to do this by looking at the average UK keyword searches for a series of topics.

Taking a range of related concepts from different levels of a hierachy we might expect to find as we move up the tree the more we find people searching with the term, because it is more general and incorporates more concepts. The theory of basic level categories would challenge this and expect the middle basic level concept to be the most used.

Look at this example taken from Google UK on the 17 July 2008.

Here the keyword dogs holds the middle ground between abstraction and detail. I think this example illustrates that search logs could be used to assist in the identification of basic level categories.

Of course this is not always the case

In this case the more abstract keyword is far more popular. This does not necessarily make it a better candidate for a Topic Page, for example it might be a difficult Topic to train due it being difficult to define.
As a rule popular search keywords indicate suitable Topics and in the majority of cases the more popular search keywords will sit at a basic level.
In the instances where super-categories are more popular consideration must be given to the extra problems this might cause the Topic Page, for example editorial overhead and awkward user navigation of the Topic as a link label.

Basic level categories are good for navigation

In addition to being good SEO practice I would also suggest that selecting basic-level categorises could improve user navigation. An example might be on contextual links where we present the name of a Topic as the link label. This assumption is based on information foraging theory.

Jared Spool has applied information foraging theory to web design with what he calls the “scent of information.” In order to efficiently and effectively forage for information on the web, seekers need to have a sense of where they are going. The design of a navigation system should provide users with an accurate “scent” that they can follow to their destination.

We have argued that basic level categories are more easily grasped than higher level categories. An example might be the categories cars (basic) as compared to vehicles (super). With the category vehicles you have a less clear sense, scent or image of what is included in this group. This could result in a lower click through rate of users from links to Topics.

Basic level catergories are good for editorial staff

Having monitored a number of Topic Pages for the last couple of months we are beginning to get a sense of what makes an easy to manage topic.

An easy to maintain Topic is:
• specific
• easily defined
• discrete

An example of an awkward concept would be crime. A Topic like this makes it hard for editorial staff to judge whether a piece of content should or should not be included. This is because it contains a list of concepts that will vary depending on who you ask and this list is liable to change over time. In the same way we talk about conceptually awkward categories for users a similar problem is presented to the editorial staff maintaining the pages.

This is apparent if you think about selecting a set of representative documents for training a Topic Page about dogs as opposed to a Topic Page about crime. As a Topic dogs is easy to train and easy to review. Content is fairly clearly about dogs or not. This is again an example of the benefits of selecting basic-level categories.

Conclusion

On reflection Topics like elected assemblies do not look like such a good idea. The individual elected assemblies as Topics would have made much better pages.

Not only are they unlikely to be searched for but they are also confusing to users when shown as contextual links. In addition they are difficult to maintain for editorial staff, again because of their conceptual awkwardness.

The following recomendations should be considered when selecting a suitable topic for an aggregation page.

Pick Topics that sit at a basic level between detail and abstraction. These concepts can be identified through the following criteria:

• frequently used in keyword searches
• represent a discrete concept
• are easily defined