Monthly Archives: June 2010

Interface Systems Evaluation & Innovation

Posted on by
Reply
The Sugar Interface

The Sugar Interface, created for the OLPC Project

I recently came across a paper discussing the evaluation of user interface systems. In it the author proposes that complex user interface systems and architectures do not readily yield to the research methods we currently use. It was at this point I started to bristle with derision in a very defensive

“I’m a research scientist and the scientific method says that we must have objective measures to express an accurate understanding!”

… kind of a way. Interestingly, this work is actually a place holder for a panel session conducted at UIST 2007 and so is therefore more discursive, with an intention to some extent to produce a reaction, such as mine, from CS facing Human Factors researchers. But, the thread is picked up again [2] in a peer reviewed submission to CHI 2008.

Briefly, Olsen suggests that complicated interactive systems are not amenable to standard evaluation methods and describes this in terms of: the usability trap / systems give inconsistent results because they require the user to “walk up and use” (as opposed to learn by experience), they use standardised tasks, and require a task completion time of between one and two hours; the fatal flaw fallacy / which suggest that evaluating fatal flaws in systems research is not possible because it would take a very large amount of time for a small team to check each possible flaw; and legacy code / which suggests that the requirement to support legacy code is often a barrier to progress. Further, Olsen goes on to suggest that instead of direct evaluation we should couch our systems development in terms of key factors such as: importance, if the problem has not previously been solved, if the solution is generalisable, if the tool assists in solution development, empowers new design participants, or if it is found that there is some enhanced combinatorial effect.

Olsen makes some quite interesting points here even though in some cases the argumentation switches between developers and end users, and the aspects which Olsen considers key such as importance etc are subjective; what he considers important I may not.  It seems to me that Olsen’s arguments suffer from a mismatch in granularity and a conflation of both the engineering and scientific domains (which is often common in computer science). At the high level Olsen is, in my opinion, absolutely correct – visionary interfaces and interactive systems cannot be tested without them being initially created, indeed the Web is a prime example of this. The first papers on the World Wide Web were rejected from the Hypertext Conference, even though it was a working system, because they did not show a clear improvement over other closed world systems such as Microcosm. However, its importance and validity were supported once the system was deployed. In reality this was a visionary idea made real by engineering  but in my opinion it was not scientific research because Berners-Lee’s work was not focused on addressing our understanding of why or how the Web was ‘better’; these being two of the primary concerns of Science.

This set me to thinking as to whether new and innovative interfaces such as “sugar“, an interface I really like – and the interface designed to be used with the One Laptop per Child project, would fare at a research conference or under scientific peer review. In reality, this comes down to the arguments made with regard to the system. In the case of sugar if the design is influenced by existing work in the field then this obviously needs to be discussed within any research paper created, it seems to me that this would be enough to justify its incorporation into any research proceedings focused on  interactive systems science and technology. However, without this well found viewpoint the justification for discussing a system within a research setting seems to be lacking. Next, claims for the system need to be discussed, with regard to sugar, the creator suggests that it is designed to assist learning and collaboration as well as communication between children. If this claim is made then there needs to be some way for it to be supported regardless of whether the system is actually based on previous third-party work. This is the point at which an evaluation must be made with specific reference to the granularity of the claims of the system’s creators.

It seems to me that Olsen’s panel paper is an interesting first step towards understanding how we can assess the contribution of large-scale interactive systems when our evaluation tools are necessarily based on often small-scale or incremental additions to knowledge. Some of the rationale proposed and the use cases discussed need further elaboration and input from third parties, interested in this research domain, would be only beneficial. However, if the work is based on well found empirical third-party research then its expression as a system can only be applauded. If the system is based on tacit knowledge or the authors “belief” then the system has little to differentiate it from any other subjective opinion and so its merit as a research artefact could not be well supported. In this case, there is a danger of stifling innovation , as was highlighted in the opening keynote of this years HT Conference [3] and resulted in some rapidly created panel sessions and BoFs. However, in the context of science and engineering and the conflation of the two in computer science, I’m not really sure how we accommodate both unless we have tracks specifically for each at the same venue but with different review criteria?

Postscript

I’ve been ‘banging around’ this post for the last couple of months on and off and just today (28th June 2010) found this nice post on ‘Woo Fighters‘ via Research Blogging Psychology and based on [4]. In this case both papers [1, 2] discussed above encourage some tenancies highlighted as possibly unscientific, in the light of [4]. However, I see this as the obvious divide we suffer in Human Factors work between creation (engineering) and discovery (science).

References

ResearchBlogging.org

  1. Olsen,Jr., Dan R. (2007). Evaluating user interface systems research UIST ’07: Proceedings of the 20th annual ACM symposium on User interface software and technology, 1 (1), 251-258 : 10.1145/1294211.1294256
  2. Greenberg, Saul and Buxton, Bill (2008). Usability evaluation considered harmful (some of the time) CHI ’08: Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, 1 (1), 111-120 : http://dx.doi.org/10.1145/1357054.1357074
  3. Dillon, Andrew (2010). As we may have thought, and may (still) think HT ’10: Proceedings of the 21st ACM conference on Hypertext and hypermedia, 1 (1), 1-2 : http://dx.doi.org/10.1145/1810617.1810619
  4. Scott O. Lilienfeld, Steven Jay Lynn, Jeffrey M. Lohr, Carol Tavris (2003). Science and pseudoscience in clinical psychology: Initial thoughts, reflections and considerations Science and pseudoscience in clinical psychology Other: 1-57230-828-1

Da Vinci the Genius?

Posted on by
Reply
Da Vinci Exhibit at MOSI

Da Vinci Exhibit at MOSI

MOSI1 is currently hosting the ‘Da Vinci the Genius‘ exhibition, which is described as:

…the most comprehensive and inspiring exhibition about the man who is arguably the greatest genius the world has ever seen. Working from Leonardo’s codices, Italian Artisans have faithfully crafted interactive and life-size machine inventions. These works include the first concepts of a car, bicycle, helicopter, glider, parachute, SCUBA, submarine, military tank and ideal city to name a few. In addition the exhibit moves far beyond machine inventions alone, featuring facsimiles of Leonardo’s most famous codices, anatomical studies, Anghiari battle drawings and Renaissance art. A recent addition is the high definition recreation of The Last Supper at actual size (29 x 14.5ft) an impressive display that compliments the existing 3D animations explaining the Sforza Horse, Mona Lisa and Vitruvian Man. Da Vinci – The Genius also includes the world exclusive Secrets of Mona Lisa – an analysis of the iconic painting, conducted at the Louvre Museum by renowned scientific engineer, examiner and photographer of fine art, Pascal Cotte. Suitable for all ages, this amazing exhibition provides a fascinating insight into not only the mind of a genius; but also into the fundamental scientific and artistic principles that he discovered.

It’s a travelling exhibition so it may be coming to a town near you. I saw this exhibit a short time ago, and it set me to thinking if the exhibition quote, ‘the greatest genius the world has ever seen’, is really true?

Looking at the work on display and the catalogue of archived work not directly available – and taking into account the additional exhibits last year, loaned from the Royal Collection, and displayed at the Manchester City Art Gallery it seems to me that Da Vinci was obvious a genius ‘artist’. His attention to detail make his illustrations both technical and accurate – and this feeling for his work imbues the paintings with emotion and energy. As an Engineer, to me he seems to have the craft based approaches reasonably well mastered with many of his ideas for siege engines, arms, and machines of war, accurately designed and practically well executed. With regard to flying machines, town plans, and other aspects of more scientific based work it seems to me that Da Vinci was a competent scientist but by no means a genius. I mainly come to this conclusion because in most of the work there is no really deep understanding of why something is happening and no real transformational knowledge derived.

Now, it also seems to me that we talk of Da Vinci as being a renascence man, that his genius was bound up in part to the plurality of his understanding, indeed, that he was a polymath. However, let us consider this in the context of the divide that existed at the time. This divide can be best summarised as having ‘Art’ on one side and ‘Natural Philosophy’ on the other, however these two domains had very different meanings than they do today. Previously ‘Art’ was meant to indicate ‘everything created by man’ while Natural Philosophy meant ‘the study of everything created by God’. When we take into account these definitions then Da Vinci was certainly and artistic genius, the domains of painting, sculpture, and engineering existing all together and not as distanced as we see them today. Further, Da Vinci’s drawings of anatomy are great studies of Natural Philosophy (as God made man) but do these alone suggest genius?

In my opinion, no; it seems to me that the pursuit of understand – of ‘why?’ (not just ‘how’) is the mark of a truly great scientist in this regard. Da Vinci seems very focused on the ‘How’ part but without much comprehension of ‘why’. So, an artistic genius without doubt, a great polymath -  absolutely, a genius scientist – not really.

Footnotes

  1. Without doubt I think that the Museum of Science and Industry at Manchester (UK) is one of the best industrial museums in the world and further, that it is *the* best of its type.

Model-Based User Interfaces and the Web

Posted on by
2
An Ontological Representation of the Internet (Credit: Bill Cheswick, Lumeta Corp.)

An Ontological Representation of the Internet (Credit: Bill Cheswick, Lumeta Corp.)

Interesting ideas coming from the Model-Based UI XG W3C Incubator Group with their Final Report of 04 May 2010 proposing model driven approaches to Web interface creation, and Web application Interaction. Within the Web Ergonomics domain I’m particularly interested in the sections on user modelling [1] via the use of an ontology [2]. Now while I think this particular approach shows a fundamental misunderstanding of the ways in which ontologies are used within the semantic Web and Description Logic communities; mainly because they seem to want to try and model an individual user as opposed to a generic type of user. I still think the ideas may be useful in that enabling simple data to be added by individuals, such as psychologists, and then a reasoned representation, with a greater expressibility derived from these ‘fragments’. This maybe just what we need to start moving deep cognitive understandings of the Web Ergonomics of users into a form that can help us simulate and apply this knowledge to instances of Web Interactivity, sort of a CogTool on steroids.

As yet, I’ve no idea how these simulation concepts could or should be linked – but we may want to look at the ongoing work in medical bio-ontological systems to see if there are some lessons (both good and bad) that we can learn from.

ResearchBlogging.org

  1. Dominik Heckmann, & Antonio Krueger (2003). A User Modeling Markup Language (UserML) for Ubiquitous Computing LNCS User Modeling 2003, 1 (1) : 10.1007/3-540-44963-9_55
  2. Dominik Heckmann, Tim Schwartz, Boris Brandherm, Michael Schmitz, & Margeritta von Wilamowitz-Moellendorff (2005). Gumo – The General User Model Ontology User Modeling 2005, 1 (1), 428-432 : 10.1007/11527886_58

‘As We May Think’ at 65

Posted on by
4
Vannavar Bush

Vannavar Bush

One of my ‘A History of HCI in 15 Papers’

Vannevar Bush (1945). As We May Think ATLANTIC MAGAZINE (July)

At 65 ‘As We May Think’ has reached its pensionable age but as yet is showing no signs of retiring. As a researcher within the Web Science and Hypertext domains, and as a member of SIGWEB, I already know who Vannevar Bush is and understand the importance of his work to my field. I know that he proposed the MEMEX, an analogue information system using microfilm which is seen as the distant forerunner to our current hypertext systems, and indeed, the World Wide Web. Luminaries such as Theodor Holm Nelson and Doug Engelbart have credited Bush as being the inspiration for their own work in hypertext and distributed systems such as the Web. So what more can we say about Vannevar Bush and the work which spurred the creation of the Hypertext field. It seems to me that Bush’s main ideas have never been more important than they are today. Indeed, the idea of association and its importance for the automation and augmentation of human thought and cognition still has resonance even today. Indeed, these concepts and the solutions they imply have become even more important than when they where first conceived, and throughout the 50s and 60s.

What influence did ‘As We May Think’ have on you? Comment at http://aswemaythinkat65.info/

It’s 65 years since Vannevar Bush’s visionary paper ‘As We May Think’ was first published in the Atlantic Review; spawning a new field of study and enthusing a generation of computer scientists, writers, and artists. The enthusiasm for the work continues today within many hypertext and hypermedia communities and we are seeing a resurgence of the hypertext ethos in the Web 2.0. Some may think that the ideas and concepts postulated in ‘As We May Think’ are old, tarnished, or without much value. But, it seems to me that the idea of association and its value for augmenting human understanding and cognition continues even today. So what of the man, what of the paper, and why does this work still continue to resonate 65 years after it was first published?

Differential Analyser

Differential Analyser

I’m looking at a picture of a bespectacled 35-year-old archetypal academic surveying the room of rods, gears, pulleys and rotors, all mechanically interconnected together; this is Vannevar Bush surveying his Differential Analyser. Bush constructed a differential analyser in 1927 while at MIT; it was an analog computer that could solve differential equations with as many as 18 independent variables. The analyser was a mechanical device which solved differential equations by integration, using wheel-and-disc mechanisms to perform the integration. It was one of the first advanced computing devices to be used operationally. It seems to me that this intimate understanding of solid physical networks, force supplied at one point generating changes and unintended consequences at another, and of physical models of abstract mathematics, defines the way that Bush thought. These real-world connections in the form of the differential analyser suggest to me that Bush understood very well the complexities and associations that are manifest in the real world. Bush’s development of the analyser and his refusal to move to digital technologies, even when those digital technologies were being developed at the same institution and at the same time, suggests that his mindset was very much concerned with quantifiable physical linkage and change. It seems obvious to me that he proposed in MEMEX, a system which also relied on the physical analogue, as opposed to upcoming digital technology.

‘Consider a future device for individual use, which is a sort of mechanized private file and library. It needs a name, and to coin one at random, ‘MEMEX’ will do. A MEMEX is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory.’

This idea, first proposed by Vannevar Bush in his 1945 Atlantic Monthly article ‘As We May Think’, is credited with being the inspiration, and precursor, for the modern World Wide Web. But for most of his article, Bush was not concerned solely with the technical aspects of his MEMEX system. Instead, as with most computer visionaries, he was more concerned with how the computer system and its interfaces could help humanity. He wanted us to understand that instead of fitting into the way a computer interacts and presents its data, the human cognitive and interactive processes should be paramount. In short, the computer should adapt itself to accommodate human needs; not the reverse.

‘The human mind . . . operates by association. With one item in its grasp, it snaps instantly to the next that is suggested by the association of thoughts, in accordance with some intricate Web of trails carried by the cells of the brain. It has other characteristics, of course; trails that are not frequently followed are prone to fade, items are not fully permanent, memory is transitory. Yet the speed of action, the intricacy of trails, the detail of mental pictures, is awe-inspiring beyond all else in nature. Man cannot hope fully to duplicate this mental process artificially, but he certainly ought to be able to learn from it. In minor ways he may even improve, for his records have relative permanency. The first idea, however, to be drawn from the analogy concerns selection. Selection by association, rather than by indexing, may yet be mechanised. One cannot hope thus to equal the speed and flexibility with which the mind follows an associative trail, but it should be possible to beat the mind decisively in regard to the permanence and clarity of the items resurrected from storage.’

The key point here is the idea that human understanding and cognition can be enhanced by tailoring information, and the associations between this information, to the way that the human brain is organised; as opposed to rigid hierarchical structures which are often arbitrary and built using abstractions such as numbers or characters. Bush realise that linking information by association has far more in common with the human brain than do homogeneous hierarchies. In reality, we move between pieces of information not based upon any abstract property of that information but by properties of how we experienced it. Bush realised that if we could continue these associations through complex information we may very well understand it far more easily and in fare more detail.

We see now that the Web provide us with massive amounts of information, but this information is difficult to get a grasp on. In this case we revert back to standard abstract types, ordering it into hierarchical structures such as Websites, alphabetised lists, lists based on date and time, or lists of search terms or tags. We can see that there is good reason for this especially when designing for a population, as opposed to an individual, however this misses the key point of association. It seems to me that in the personalisation and adaptive hypermedia domain there is far more scope to personalise the experience of the information we gather as opposed to the use of purely static structures, or those of the Web 2.0 and Ajax; which are often event related.

In ‘As We May Think’, Vannevar Bush showed us a vision of how information could be manageable even on a large scale by association based on the experience of the individual. And further that this association would enable individuals to better comprehend information and thereby enhance our intellect and enable us to better investigate the world around us.

Bush is quoted as saying that, ‘Science has a simple faith, which transcends utility. It is the faith that it is the privilege of man to learn to understand, and that this is his mission.’ Personal associations derived from experience, and not abstract structures, enables us to learn better and understand better, in this way Bush’s faith in science and humanities mission can be better accomplished.

ResearchBlogging.org
REFERENCES

  1. BUSH, V. 1996. As we may think. interactions 3, 2, 35–46.
  2. LEVY, D. M. 2005. To grow in wisdom: vannevar bush, information overload, and the life of leisure. In JCDL ’05: Proceedings of the 5th ACM/IEEE-CS joint conference on Digital libraries. ACM, New York, NY, USA,281–286.
  3. OINAS-KUKKONEN, H. 2007. From bush to engelbart: ’slowly, some little bells were ringing’. IEEE Ann. Hist.Comput. 29, 2, 31–39. OWENS, L. 1991. Vannevar bush and the differential analyzer: the text and context of an early computer. 3–38.
  4. SIMPSON, R., RENEAR, A., MYLONAS, E., AND VAN DAM, A. 1996. 50 years after “as we may think”: the brown/mit vannevar bush symposium. interactions 3, 2, 47–67.
  5. WALDROP, M. M. 2001. The dream machine: J. C. R. Licklider and the revolution that made computing personal. Viking, New York.
  6. ZACHARY, G. P. 1997. Endless frontier: Vannevar Bush, engineer of the American Century. Free Press, New York.