Sediment

Current thought conceptualises broad¹ data as one layer of data overlaying another usually onto some base-data; the base-data being useful and referable to the user. These base-data may be absolute geographic coordinates, relative artefact coordinates — being eye-tracked say, or navigation pathways through websites. The key feature is that the base-data is a common dimension among the datasets which will be layered, and are relevant to the area of inquiry.

However, thinking in layers makes the computational work easier for computer scientists but misses important information found when layers interact with layers. This interaction is often missed, but always occurs in real world scenarios. And so the way we describe the world by interactions between artefacts is lost.

So I’ve been thinking of two new concepts. The first is the idea of ‘Information Sediment’ to describe the process by which layers of data interact with other layers of data, at the interface of both. In this way we can discuss soft flexible bands of data as opposed to the hard layers we currently have.

In the real world interplay between entities — that the data represent — always occurs. Limiting this to fixed layered groups of data does not give the realism that will be needed in a big data world. The key aspect here is granularity. To be able to describe this better we need to stop thinking of data as holistic sets, fixed once collected and processed, but as flexible entities which can interact with other datasets. One way to do this is to think of data as particles — ‘Data Particles’. In this way we can create algorithms which allow data layers to influence each other to generate new results, all tied to the common component present in all data — the base-data.