Those following the Collection Improvement PoC might be interested in the edna Content Standards overview [also a PDF version 213Kb], - this describes policy issues that the PoC team are trying to automate.
In order to help people to get away from ‘production think‘, I’ve been struggling with trying to precisely define a Proof of Concept. It’s almost easier to define what it’s not…
Back in March this year, I thought:
A high risk, high trust and low governance project that creates a conceptual solution for the client. Focus solely on the concept. Forget security. Forget firewalls. Forget QA etc. What is delivered is a bare-knuckled prototype which demonstrates how it could solve the issues your client deals with.The project has no limits other than Time (approx 3 months) and Resource a small team working part time within a small financial budget. Concluding each POC month is a ‘Show and Tell’ to an audience of sponsors, users and internal teams - each one being more public than the last.
Not satisfied this with mouthful, I had another go in early October:
“A practical process designed to discover and focus on the core of what and how to innovate.
By placing people in a deliberate environment of trust without real-world constraints other than limited resource, timelines and lightweight project management, a PoC nurtures innovative creative solutions.
Chatting with Mike Seyfang - he would also chip in:
- a vehicle for embracing risk
- an environment that creates space in work for play “expect 6/10 to ‘fail’ but 3/10 might just be ‘hum dingers’
Then again he had the urge to spurt out:
3 people - 3 months - $30k
…there’s a certain appeal in the above’s simplicity.
What’s your thought’s about trying to define an innovation process?
“We want to buy it” was one reaction from a guest at Show and Tell 2.
This is a common reaction to most Show and Tells; something which I take a sign of success (we’re on the right track) yet cringe with the thought ‘no - it’s a PoC; a prototype rather than a product‘.
Judging by the number of good questions, the audience were engaged in the topic and saw potential in what was been demonstrated.
Our reaction to how Show and Tell 2 went, the team came to some conclusions:
Today’s blog post comes courtesy of Sarah
Show and Tell – although most of NCVER people can’t come we will go ahead on Fri 28th. Vaughan will invite the Flinders people. We will take 30 min, from 4 to 4.30, followed by drinks. Sarah will send general invitation to ed au and book Jo’s Café. We discussed what we will show – see 3 below.
Pru reported she has started looking at the data that Nick sent from Edith – approx 2000 records. She has put into an access database with fields to enter decisions and comments. She and Sarah have developed some criteria for rejecting records; the idea is a record will be rejected outright, based on edith fields only, or then sent to IOs for human perusal and evaluation. This is a research exercise to see e.g.
a) can a valid rejection decision be made like this?
b) What criteria are sensible to use?
c) What other rules occur to us as we look at the data?
d) Will different people make the same decision?
e) How would a machine do in comparison?
Pru, Sarah and Nelly will send Vaughan the suggested rejection criteria. We decided that Sarah, Pru, Nelly and Vaughan will look at 100 edith records that Nick will supply. If possible we would like 50 coming from taggers and 50 coming from tags, otherwise as random as possible. They will be sites that are not already in edna. We can also compare a machine that looks at the content of the web site to the human evaluation of the edith record only and the machine evaluation of the edith record only. We can also look at the delicious tags (Nick may be able to harvest these for the 100 records or else we will do it manually, taking a segment each) and see whether the delicious tags would have made a difference to the evaluation decision (or will they only be useful in the 3rd stage of metadata creation).
(Sarah queried what happens if someone has reset the quality vs quantity dial – are all the records streamed from edith changed until the dial is changed again? The answer is yes (there is no “universal” data stream that continues to be harvested). We agreed this might need to be different in a production/implementation of the idea but for the proof of concept at present it doesn’t matter.)
3. For the Show and tell we will:
4. Next meeting
We agreed to meet on Tuesday as Thursday won’t give enough time to prepare the presentation. Nick can’t be there but most of the technical work for this part is Vaughan anyway.
Time to get the elves and the shoe maker busy again.
Required research: (a gem via Vaughan)
“A valuable way to gain understanding of a complex network is through the identification of its important, or prominent, nodes.”
To get to Show and Tell 2:
To get to Phase 3 ‘Describe’:
Yesterday’s meeting touched on the benefits of a strong vision. The vision provides a yard stick to measure all ideas against.For the evaluation stage, I started rambling on about ‘there should be an option for a IOs to tick if the resource is an event or document and therefore present a different set of metadata fields’.
Wait - remember the vision says:
“Collection improvement through user engagement and better metadata tools”
EDITH is a metadata Tool which is principally
So….. reflecting on this, EDITH should not require IO intervention for a result to be produced. Therefore it does not require a user interface. It simply provides a list of resources with metadata that can be exploited using other collection management tools such as DSpace, RSS, etc.
Click! Ahhhh! EDITH is a resource metadata automation tool. Any sort of manual intervention in the flow of the process should be eliminated.
This gets us to a simpler, more elegant and closer to what’s really needed.
Thus the vision keeps us honest and guides us along the right POC path.
Through EDITH, your candidate educational website will be in the company of a lot of others: currently EDITH is generating over 250 website candidates a day!
With so many websites, which ones should Information Officers add to the collection?
In last week’s POC meeting we realised that the existing EDITH selection path for candidate website would not best reflect the edna collection policy.
Yes it would filter out the rude sites and non-english based resources - however it ignores other collection policy criteria such as focusing on Australian based materials.
We will seek to rank candidate website based on the collection criteria: for example:
Applications:
Extensions:
In other thoughts:
Last month I challenged you to think how we could describe website and online resources accurately.
Here’s our thinking for the metadata ‘describe’ phase: to turn describing websites into a game that anyone can play online.
Here’s how we got there:
1. http://www.mturk.com/mturk/help?helpPage=whatis Mechanical Turk is probably one of the best known early web based crowdsourcing systems used to get people to describe (metadata) that is normally very difficult for a computer to do well.
2. http://recaptcha.net/ is a cleverly combines completely different tasks
a) checking that a human is completing a online form (and not a computer)
b) improve the accuracy of digitising of books through character recognition
What’s interesting here is how it capitalises on a moment of human skill (recognition) that is ordinarily being wasted on a common website verification task (CAPTCHA) that is occurring on the scale of the internet.
3. http://images.google.com/imagelabeler/ is a simple game that is strangely addictive, helping improve the accuracy of image description.
I got very excited about how the structure of the web is changing under the momentum of web 2.0. The penny dropped when I realised Flickr, YouTube, Google and Del.icio.us, Yahoo Pipes aren’t websites - they’re fundamental web 2.0 service infrastructure.
Those organisations who doing great things in the web2.0 space are those who do one thing and doing it very well. The service often is characterised by openness: empowering everyone to put stuff in, describe and get stuff back out in order to mash, learn, share and collaborate.
It used to be considered poor form for websites to sponge media from other website servers. Flickr, YouTube, Google, and Del.icio.us (including RSS and website APIs) have inverted this. Now these sites positively encourage you to embed content and applications stored on their servers.
The websites of each of these are not really useful in themselves. Their true power really is their capacity to
This is why (in my mind) they are the key building blocks for the new Web 2.0 world.
What’s could be some key Web 2.0 infrastructure services for education? Some initial thoughts:
What do you think are the key Web 2.0 infrastructure elements for education?
The regular reader of this blog would know the team wants to make what’s hard about collection management easy. This week’s meeting asked the question what’s hard about meta-data.
The answer is of course is creating ‘correct’ metadata. For example how would you extract the subject out of a website, PDF, Word doc, audio file etc etc.
We could do it modifying brute force tools such as:
However! Your challenge is to tell us how you think we can do it.
We have a few clues of what we are currently thinking.
Have a look & play with:
Leave a comment on how you think we can create ‘correct’ metadata.
