Inter-lab technical replicates show a high level or reproducibility and linearity at the spot level.

The scientific method is based on empirical data and the replicability of those experiments used to collect this data by third parties. However, in Human Computer Interaction (HCI / CHI) the ability to replicate experiments is not possible because there are no peer reviewed routes for data publication. This inability to replicate experiments due to a lack of public data means that HCI does not strictly follow the scientific method, and as such is sometimes seen as not rigorous. Indeed, I’d go further and suggest that:

1. Methodical data collection and experimentation is variable because the data may never been seen by a third party;
2. Experiments to confirm the science are not repeated as the data and experimentation methodology is not available; and
3. Data collection and accurate experimentation is not seen as being important for citation and publication of the data alone, and is seen more as a second class citizen to an academic technical paper.

Other sciences (Physics, Ecology / Biology, Medicine etc. ) recognise the value of their data as it costs (often) enormous amounts of time, effort, and money to acquire. Indeed, this is the whole reason the ‘Economic and Social Data Service‘ in the UK exists.

In answer, I’d propose something like the ‘Journal of Human-Factors Experimental Methodologies, Data-sets, and Analysis’ (HEMDA) to address and rectify these problems and support the rigour of our discipline. Soliciting extended peer reviewed abstracts conforming to one of four strict templates and accompanied by publicly available electronic data-sets, I’d hope it would publish Methodologies, Data-sets, Comparative Analysis Techniques, and the results of Experimental Repudiation. Following the style of the British Medical Journal, I’d also wish to provide electronic links to related articles and enable rapid responses from the community. In this way experimental data and its collection becomes rigorous, technical papers to other journals will have the strength of the data citation to HEMDA, data can be used for a purpose other than the one it was collected for, and research arguments are strengthened.

Hopefully, someone will agree with me and create such a Journal!