Research Funding – and a Happy New Year 2010

Time Square New Years Eve!

Time Square New Years Eve!

Well first off, let’s say goodbye to 2010 and welcome in 2011 – I’m sure Time Square will be as crowded as it was in the 1950′s – different but the same!

Now lets look at research funding [1] – I think this can be equally applied to paper acceptance rates – in the hope of a better funded 2011! Current thought seems to be that 30% is about the right level of acceptance for funding. Below 15-20% it seems that you will miss some good work, and get some bad work – at 30% you’ll get all the good work and some bad work. The bad work doesn’t necessarily rise that much with the 30% threshold either. To quote the article:

Reviewers say that they feel forced into making impossible choices between equally worthy proposals, especially when success rates are less than 20%. “That’s in a range where you have lost discrimination,” says Dick McIntosh, professor emeritus of cell biology at the University of Colorado in Boulder. “That’s a situation where you are grading exam papers by throwing them down the stairs.” The chairman of the ACS panel agrees. “Deciding between the top grants, I don’t want to say it’s arbitrary, but it’s not really based on strong criteria,” he says. “It’s subtle things.”

From: Research funding: Making the cut. Nature, 467 (7314), 383-5

From: Research funding: Making the cut. Nature, 467 (7314), 383-5

But of course a 30% funding level costs more than a 20% one. This is the problem as we can see in many sectors (here elaborated in the graph below we can see this represented for the USA’s National Institutes of Health figures for 2000 to 2010) funding levels are falling. Indeed, to quote the article:

At the National Institutes of Health (NIH) in Bethesda, Maryland, which funds the majority of biomedical research in the United States, several years of flat federal funding combined with a rise in the number of applications means that 21% of research-project grant applications were funded in 2009, down from 32% ten years earlier (see graph). The situation in many other countries is just as tough.

It seems then that we are setting yourselves up to fail in the future. If the decline in funding continues we will start to loose critical research because it is too edgy, too visionary, too forward thinking – just the kind of work that falls into the 20-30% band. Good progressive work is needed but those that fall into the <15% acceptance band are often safe projects with some little novelty or that propose ideas which do not trouble or challenge the reviewer.

Disclaimer: Now all this could be ‘sour-grapes‘ as most of my work seems to hit at 22-25%!

References

[1] Powell K (2010)Research funding: Making the cut. Nature, 467 (7314), 383-5. PMID: 20864969

Addendum – 06th June 2011

Nice piece on paper acceptance rates at 30% – ‘Conference Paper Selectivity and Impact

Defining UX – and a Merry Christmas 2010!

Merry Christmas 2010

Merry Christmas 2010

As a positivist research scientist I’ve been struggling with the whole User Experience (UX) space for a long time, because to me it just seems a bit – well – ‘fluffy’. Many people seem to have got to grips with it, including Rui Lopes (see his recent blog articles) but to me, the more I read about the subject the more I think it is fine for evaluating specific interfaces but that the results cannot be generalised. It seems to me that it allows us to say something about – the often intangible – aspects of interfaces and their design, but it doesn’t help us say much about interfaces in general, or provide an qualitative underpinnings that are replicable for different interfaces. To me it is useful to give the gist or the flavour of the experiences of a specific user group or individual and a specific interface.

In the past I’ve suggested that User Experience (UX or UE) is often conflated with usability but takes its lead from the emerging discipline of experience design (XD). In reality, this means that usability is often thought of as being within the technical domain. Often being responsible for engineering aspects of the interface or interactive behaviour by building usability paradigms directly into the system. On the other hand user experience is meant to convey a wider remit which does not just primarily focus on the interface but other psychological aspects of the use behaviour.

In reality, user experience is very similar to usability, however, it found its roots within the product design community as opposed to the systems computing community of usability. The usability specialist would often be expected to undertake a certain degree of software engineering and coding whereas the user experience specialist is often more interdisciplinary in focus. This means that the user experience specialist may undertake design of the physical device along with its economic traits but may not be able to take that design to a hardware or software resolution. Indeed, user experience has been defined as ‘pertaining to the creation of the architecture and interaction models that impact a user’s perception of a device or system. The scope of the field is directed at affecting all aspects of the user’s interaction with the product: how it is perceived, learned, and used.’ therefore user experience is less concerned with quantifiable user performance but more the qualitative aspects of usability. In this way it is driven by a consideration of the moments of engagement’, known as ‘touchpoints’, between people and the ideas, emotions, and the memories that these moments create. This is far more about making the user feel good about the system or the interface as opposed to purely the utility of the interactive performance.

User experience then, falls to some extent outside of the technical remit of the computer science trained Human Factors specialist. However, it is likely that Human Factors specialists will be required to work with user experience, or experience design specialists, and this is far more likely to be the case with Web or mobile focused developments. However, because the user experience specialist is likely to be from a design background, scientific and therefore summative evaluation may not lie within their skill set.

However, I’m pleased to see I’m not so far from the ‘pack’ as I imagined. In their paper [1] Law et al. interview 275 researchers and practitioners from academia and industry, and it turns out that most respondents agreed that UX is dynamic, context-dependent, and subjective. The majority strongly agreed with the statements ‘Fleeting and more stable aspects of a person’s internal state (e.g.. needs. motivations) affect a person’s experience of something’, ‘UX occurs in. and is dependent on the context in which the artefact is experienced’, and ‘Prior exposure to an artefact shapes subsequent UX’; with a majority also weakly agreeing to ‘Designing (for) UX must be grounded in user-centred design’, ‘UX can change even after a person has stopped interacting with the artefact’, ‘UX is based on how a person perceives the characteristics of an artefact. but not on the characteristics per se’.

So it seems that there is broad agreement that UX is as more about a single specific user experience – an experience that may also change – than it is about a generalisable result.

References

  1. Law, Effie Lai-Chong and Roto, Virpi and Hassenzahl, Marc and Vermeeren, Arnold P.O.S. and Kort, Joke (2009). Understanding, scoping and defining user experience: a survey approach Proceedings of the 27th international conference on Human factors in computing systems, 1 (1), 719-728 : 10.1145/1518701.1518813

W4A Paper Deadline is ‘Danger Close’ – #accessibility #a11y #w4a11

Web Conference Logo

Co-Located with WWW2011 in Hyderabad, India.

So we are coming very close to the W4A Paper deadline for the 2011 edition, indeed, it is on the 10-Jan-2011. The theme this year is ‘Crowdsourcing the Cloud: An Inclusive Web by All and For All?’. While Crowdsourcing the Cloud is the theme, please don’t be deterred if this somewhat unique area is not yours. The organisers would like to see all quality work on Web Accessibility regardless of the particular field within accessibility. The overriding reason for a paper being accepted is its high quality in relation to the broad area of Web Accessibility. This years W4A is endorsed by the IW3C2 in cooperation with the ACM and its Special Interest Groups SIGWEB and SIGCHI. The general conference is supported by IBM Research, Microsoft, and Google, while the Web Accessibility Challenge is specifically sponsored by Microsoft, and the Student Awards are specifically sponsored by a Google. You can also find them indexed in the ACM DL and DBLP, and on Twitter, Facebook, LinkedIn, Lanyrd, Flickr, SlideShare, and Interaction-design.org.

I’m also pleased to see the fees – $250 for Students and $350 Professionals – are pretty reasonable for a two day international conference. What’s more I’m super pleased at the ACM Bibliometrics: publication history giving, very slightly below ACM ASSETS for citations (0.8) and 200 downloads per paper above them.

The top two papers for citations are:

  1. Vigo, Markel and Arrue, Myriam and Brajnik, Giorgio and Lomuscio, Raffaella and Abascal, Julio (2007). Quantitative metrics for measuring web accessibility Proceedings of the 2007 international cross-disciplinary conference on Web accessibility, 1 (1), 99-107 : 10.1145/1243441.1243465
  2. Bigham, Jeffrey P. and Ladner, Richard E. (2007). Accessmonkey: a collaborative scripting framework for web users and developers Proceedings of the 2007 international cross-disciplinary conference on Web accessibility , 1 (1), 25-34 : 10.1145/1243441.1243452

Both 2007 papers with 19 and 18 citations respectively – which seems to suggest the W4A is a slow burning conference for work that hos long term scientific value.

Full Disclosure: I may be biased as I did help set up this conference. But I think the figures and the Microsoft Academic Search Conference Trajectory speak for themselves.

Publication years 2004-2010
Publication count 157
Citation Count 374
Available for download 154
Downloads (6 Weeks) 2,000
Downloads (12 Months) 15,627
Downloads (cumulative) 66,917
Average downloads per article 434.53
Average citations per article 2.38

Accesskeys Still an Issue – #accessibility #a11y

Excesskey Extension Screenshot

Excesskey Extension Screenshot

(X)HTML Accesskeys have existed for years, and through most of this time I’ve been pushing for WebApp / RIA centred accesskeys, or in reality accesskeys linked – not by a character – but by the semantics of the functionality that the key relates to. Just recently, Charles McCathieNevile (chaals) who works for Opera Software Standards Group has revisited the problem by creating an accesskeys extension for Opera 11 beta — documentation is available. You can see  our conversation and the extension features in the public W3C WAI UAWG Mail Archive starting with Chaals initial post and continuing with mine. In general the upshot of my view is that:

Maybe a taxonomy of common functionality would help here – “Save”, “Save As”, “Save Preferences” would all be in there but accesskey=”Save the Whales” would not. I’m sure this level of common functionality would be doable – indeed most OS come with common functionality added to the menu widget. In fact it could be cooler than that, such that if Opera were being used on mac it would convert “edit options” to “edit preferences”? If I make a user assignment that “edit preferences” are always [designated_browser_modifier-key]+p then I would always get this.

While I agree with Chaals that the ‘rel’ attribute can be used – I think accesskeys with a meaning from a common taxonomy will enable far better control of WebApps or RIA. Lets hope development continues and Chaals decides to implement something like this in the future.