Monthly Archives: July 2010

Reproducibility is Key to Human Factors Science

Posted on by
4
Reproducibility

Inter-lab technical replicates show a high level or reproducibility and linearity at the spot level.

The scientific method is based on empirical data and the replicability of those experiments used to collect this data by third parties. However, in Human Computer Interaction (HCI / CHI) the ability to replicate experiments is not possible because there are no peer reviewed routes for data publication. This inability to replicate experiments due to a lack of public data means that HCI does not strictly follow the scientific method, and as such is sometimes seen as not rigorous. Indeed, I’d go further and suggest that:

  1. Methodical data collection and experimentation is variable because the data may never been seen by a third party;
  2. Experiments to confirm the science are not repeated as the data and experimentation methodology is not available; and
  3. Data collection and accurate experimentation is not seen as being important for citation and publication of the data alone, and is seen more as a second class citizen to an academic technical paper.

Other sciences (Physics, Ecology / Biology, Medicine etc. ) recognise the value of their data as it costs (often) enormous amounts of time, effort, and money to acquire. Indeed, this is the whole reason the ‘Economic and Social Data Service‘ in the UK exists.

In answer, I’d propose something like the ‘Journal of Human-Factors Experimental Methodologies, Data-sets, and Analysis’ (HEMDA) to address and rectify these problems and support the rigour of our discipline. Soliciting extended peer reviewed abstracts conforming to one of four strict templates and accompanied by publicly available electronic data-sets, I’d hope it would publish Methodologies, Data-sets, Comparative Analysis Techniques, and the results of Experimental Repudiation. Following the style of the British Medical Journal, I’d also wish to provide electronic links to related articles and enable rapid responses from the community. In this way experimental data and its collection becomes rigorous, technical papers to other journals will have the strength of the data citation to HEMDA, data can be used for a purpose other than the one it was collected for, and research arguments are strengthened.

Hopefully, someone will agree with me and create such a Journal!

Decentralized Extensibility Worries the Bejesus Out of Me!

Posted on by
Reply

Decentralised ExtensibilityNoah Mendelsohn (an IBM Distinguished Engineer) defines Decentralized Extensibility (DE) in HTML5 as:

“The ability for a language to be extended by multiple parties who do not explicitly coordinate with each other.”

The rationale for DE is that:

“The Web is too big for any central group to invent or cooordinate all needed extensions to languages like HTML”

Indeed, HTML5 supports DE:

“When vendor-neutral extensions to this specification are needed, either this specification can be updated accordingly, or an extension specification can be written that overrides the requirements in this specification. When someone applying this specification to their activities decides that they will recognise the requirements of such an extension specification, it becomes an applicable specification for the purposes of conformance requirements in this specification.”

However, DE has been causing a lot of polarisation with the W3C and the Web Community in general but as of WWW2010 it still seems to be on the cards. Now it seems to me that supporting DE is going to be incredibly difficult, incredibly prone to errors, and incredibly easy to miss-use.

To me this seems to be a way for the big browser manufacturers to cut out the W3C process entirely while still conforming to W3C standards; standards which these manufacturers created. But in reality how can we expect specialist browsers and user agents, and extensions and plug-ins to support these extensions. Indeed, there is no requirement for full specifications of the underlying semantics or functionality to be published. Yet again we allow the possibility of descending back to multiple closed models of the Web as opposed to the open one maintained by universal standards. The W3C process maybe slow but that slowness allows everyone to catch up with the big players who have massive resources to throw at user agent development.

From the authors prospective, why would I write general content using specific extensions only supported by a single browser. The big manufacturers must already know this , and so it seems to me that they are either: (1) gambling on cornering the market and forcing everyone to write for their specific browser; (2) creating the ability to use extensions to their own content which hook into their own browser, or (3) usurping the W3C Standards process. All these seem bad to me for both accessibility and Web ergonomics in general.

But we already have this in the form of microformats, I hear you cry! Microformats have been bubbling under for two to three years, mainly being adopted by researchers and Web specialists, there has not yet been widespread adoption into the general Web building community. They are built using existing standards compliant technology and are intended to solve simple problems without the need for full language and technology redesigns. The formats created are meant to be read by humans (as opposed to semantic technologies) and follow an open and readable format; I think of microformats as RDFa Lite. This means that any application which can understand this microformat could add different presentation and interaction modalities to pages containing any defined microformat. But microformats suffer in similar ways to DE formats in that they are inconsistent between browsers, the lack of a universal standard means most browsers, sites, and proxies don’t handle them, and those that do handle them differently.

I see microformats as an incubator for new ideas which should be included in the next versions of the technology. In this case, the solution to handling DE is to centralise it under a fast W3C Recommendation addendum procedure. This continually chartered activity would look to move extension inclusion submissions to a recommendation addendum in, say, six months. In this way we can get fast turn around of extensions but ones that fit into a universally agreed and implemented specification.