GROK-IT
SCIENCE 101
Physio-Info-Tronics
for Perceptualization Environments:
An
Anthrotronic Interface System for the Emerging Information-Communication Matrix
Abstract
Advanced research in the area of sensors and information systems continue to produce an emerging array of networked information and communication (info-com) technologies. These in turn produce an ever-increasing volume of information that must be perceived and understood by a human operator. We propose to develop an interface to the network through which human communication of and interaction with this information is facilitated. It is postulated that an optimal mapping of interactive interface technologies to the human nervous system's capacity to transduce, assimilate and respond intelligently to information in an integrative-multisensory interaction will fundamentally change the way that humans interact with information systems. Therefore, this research is involved in studying the flow of information between the human and the computer. The intent of this effort is to identify methods and techniques that optimize information flow between humans and computers. From the philosophic orientation of General Systems Theories and Physiologically valid human information sciences we are exploring relevant issues in interactive human computer interface design. An eclectic integration of cognitive neuroscience, perceptual psycho-physics and bio-cybernetics guides our efforts to create robust interactive systems. This paradigm of interface technology is based on new theories of human-computer interaction that are both physiologically and cognitively oriented. Optimization of human interaction with vast assets of networked systems information incorporates multi-sense rendering technologies, giving sustained perceptual effects, and incorporates other natural user interface devices that measure multiple physical and physiological parameters simultaneously and use them as inputs. Such a biologically optimized interactive information interface method offers the potential of hyper-effective communication. This increase in effectiveness will impact both human-computer and human-human communication through extended perceptual dimensionality and “enhanced expressivity.”
http://www.medibolt.com/gb2k/flash/introduction.html
Introduction
The amount and diversity of information being produced in the many fields of human knowledge are growing rapidly. Whether it be academic researchers creating and publishing new information based on already existing material they have used in the development of new theories and methods, or the information that is generated, used and recorded during more sudden and sporadic occurrences of emergency medical response, for example, the volume and differences in information increase. Complicating this state of affairs are the integrations of information across boundaries and events made possible by emerging communications technologies. Cross pollination and syntheses of the diverse data means that there are more and more possibilities for action, when all this information is brought to bear on contexts of need. Interacting profitably with the many sources of information using the many means of communication now available requires new models of how humans may interact with information resources. In this text, our purpose is to articulate one such model. Our research into human-computer interaction, particularly as it developed conceptually with the needs of users with disabilities, has allowed for some very effective ideas on how to optimize the human’s ability to exploit the powers of information & communications systems. Anthrotronics (meaning “human instrumentation systems”) is a term we have introduced to emphasize the human as central in the design and use of information and communication technologies. The human ability to perceive, process, and act on information is directly related to the physiological structures and functions of the nervous system. The orientation of the model we propose is therefore towards a tailoring of information representation to the unique capacities of the nervous system to perceive and respond to information.
http://www.medibolt.com/conbottom.htm
The early stages of this research occurred in the neurology and rehabilitation departments of a major medical school. These have led to a new focus on perception and expression as the key problems in re-thinking how humans might better interact with information systems. Historically, these problems were sidelined by the emphasis on faster more powerful computation and greater storage. Human factors were, therefore, largely omitted from the consideration and design of information systems. Below we provide a brief background on how these issues were formulated in the particular contexts of medical diagnostics and rehabilitation. These were merely the starting points for the interrogation of the logic of human-information interaction, in which field we are now fully engaged.
Electroencephalographic (EEG) and Magnetic Resonance Imaging (MRI) devices were both used to gather clinical data from the human brain. These are the among the tools of the clinical neurologist. Electrophysiological output from the brain is what is thereby measured and output as either “squiggly lines” (EEG) or crude image maps of the head (MRI). Experimentation with electrocardiograms (EKG) was also undertaken. This research led to the conclusion that the perceptual forms of the data generated by these technologies were inadequate to the task of providing precise diagnostic information to the clinician. Further, the form of the data prevented anyone but the highly experienced clinician from making decisions concerning obscure features in the data. Based on this experience, the perceptualization of data became a general problem to be explored and addressed.
Neuro-diagnostic studies of the brain also triggered the awareness that electrophysiological output comes not just from the brain, but from the muscles of the entire body. The body is emanating certain kinds of energy as measurable patterns at all times. EEG, EKG, and other physiological monitoring instruments simply ‘read’ this energy as data, as it emanates from the body. The proximity of the pediatric rehabilitation to the neurology department, where Warner was doing this work, occasioned the question as to whether physiological output such as this could in fact be exploited as an input source. That is, if the body is giving off certain energies in certain patterns in relationship to some mental or physical behaviors, then it is conceivable to use them to control some device like a computer. The biosignal output need only be captured in a way that would allow it to be converted into an input for driving a device.
Based on these insights, the issues of perceptualization and expression within information systems were raised and shown to be problems in need of solution for a wide range of users in diverse fields. Rehabilitation and clinical/emergency medicine have been the areas in which we have endeavored to achieve both an articulation of the problem and a generic conceptual framework for its solution. More recently, we have engaged with the humanitarian and disaster response communities to show the ways in which existing practices of information gathering, data representation, and decision support are inadequate.
http://www.medibolt.com/gb2k/video/storyboard/index.html
Note that the only reason we can
make such claims is that technology
has been emerging which allows wholly different ways of interacting with information. Our
strategy has been to research these technologies and, within particular contexts, to test and refine our ideas
about how to use them. We now turn to the generic issues.
Anthrotronics constitutes human-centered and, more specifically, mind-centered thinking about information and communications systems design. Rather than beginning with the perspective of power and storage, these are included in the larger framework of beginning with the needs and abilities of particular users as a ‘mind in the system’ acquiring and outputting information. Our assumption is that current interaction with vast and diverse information resources for varying and urgent purposes is hindered by:
The visual sense has been given top priority in the area of human-information interaction. With few exceptions, data tend to be represented visually. Two other senses which are usually omitted from the information interaction: a) hearing and touch, which has seldom been exploited as a means of information gathering (exception: Braille for users who are blind). Even the way information has been prepared for the visual sense is often inferior, as it has been presented as text, numeric characters, and crude graphics. The medical expert is normally dealing either with printed textual material, or else with arcane visual forms of data produced by medical instruments like EEG and EKG. Text, numbers, symbols, squiggly lines, and graphs have perceptual qualities often inadequate to time and content requirements of those using the information (doctors, emergency medical personnel, etc.).
The information, as typically represented, is perceptually
deficient, to the neglect of the extraordinary capacity of our brain to capture
and process information from our senses. We are not saying that vision or text are bad ways of accessing
information. Rather, we are making a conceptual point that while these work
well for some needs, the critical nature of some information requirements is hindered by reliance on
them, especially when vast quantities
of diverse information must be accessed, represented in all their richness, and then used
in a rapid manner. Our goal is to offer conceptual—and concrete technological
prototypical—solutions for information demand by rethinking the sensory and perceptual possibilities for
how to render information
to the human body for decision support.
Our goal, based on these concepts, is to develop systems that incorporate diverse multi-sensory representations of information into a unified dynamic interface. The approach is based in part on concepts in sensory physiology. A mind-centered orientation to human-information interaction asks first, “How does the human nervous system, through the senses, gather raw data and then present it as information to the mind?” The answers will help us create powerful interfaces structures and functions between minds and data. Thus we are proposing to integrate the conscious human user into this system as a computational resource: a mind (not just a user) in the loop. By increasing the number and variation of simultaneous sensory inputs, we can make the body an integral part of the information system, “a sensorial combinatoric integrator.” That is, the mind and body inside the network interface we are proposing will be a locus of perception and expression: a reader and a responder in any information- and decision-intensive process.
To this end, we will identify the
optimal perceptual parameters in which information can best be rendered for
each of the three senses named above (vision, audition, and touch). That is,
what types of information
are best rendered to each specific sense modality and how can we optimize the
representation based on the unique processing properties of the sense in
question?
Research in human sensory
physiology, specifically sensory transduction mechanisms, demonstrates that there are designs in our
nervous systems optimized for
feature extraction of spatially rendered data, temporally rendered data, and textures. Feature
extraction is defined by Kandel et al.[1]
as “the selective detection and accentuation by sensory neurons of certain
features of a stimulus.” Models of information processing based on the capacity
of these
neurophysiological structures to process information will help our efforts to enhance
perception of complex relationships by integrating visual, binaural, and tactile sense perception. Then, by
using electrophysiological signals as input (see above), we can generate highly interactive systems in
which these biological signals initiate specific events. Such a real-time analysis enables multimodal
feedback and closed-loop
interactions. We will endeavor to
address and solve the deficiencies in conventional information representation
and decision support through both the perceptualization of information
and the enhancement of expressivity made possible in a single
interactive interface that can be deployed in any place equipped to benefit
from it. Information will be rendered
for three senses rather than one. The eyes, ears, and skin will all
become avenues for
gathering data, with vocal and haptic signals allowing for command inputs.
Perceptualization of information is an idea which assumes that, under high-intensity demand for
diverse and voluminous information, it is best to divide the information. After processing with data
mining techniques, the goal is to
take some of the data and put them into a form highly accessible to the visual sense.
Likewise, take some of the data and make them accessible to hearing, and similarly
with the tactile surface of the body. Large quantities of diverse data may be transformed into
multi-sensory forms of information.
Here are the basic modalities of perception and expression:
http://qube1.mindtel.com/users/projects/topper/10-11-2000_Naturala//
Data mining and processing are part of the core functionality of the interface. In order to transform data into the different perceptualizations available to the user, a powerful means of turning repositories of data into novel and powerful information is required. The following relevant quotes are from www.spss.com:
Data can, of course, come in many forms. Some are in databases and data warehouses. However, a good deal of the data, with which a user of this interface will be interacting, would be generated on the fly in crisis situations. For example, medical monitors at the site of a human emergency would be streaming raw data into the “grok-it” interface for the user to perceive and respond to immediately. Or, if physicians were doing clinical work over the Web, there would be simultaneous processing of information from databases as well as on-the-fly generation of patient data. Counteracting the effects of bioterrorism would be another example of requiring data from both large stationary stores and data coming from a vast array of different kinds of sensors at and near the site of emergency.
We propose to research, prototype and evaluate an integrative interface matrix that couples data streams from sensors, micro-informatic technologies, and databases to the mind via an intelligent exploitation of the nervous system towards the enhancement of perceptual dimensionality and expressive capacity. This anthrotronic (human-scale instrumentation system) interface matrix will allow for the harnessing of the human nervous system in ways that increase the user’s ability to “grok”[2] and communicate the information being generated and transmitted by the vast multi-domain information-communication system.
Further, we
will research, prototype
and evaluate technologies that enable controllability and exploitability of
the multichannel, multifunction concurrence of dynamically interconnectable bio-couplers to the
info-com system. The foundation for this goal is the proposition that the
information flow between external
sources (representation) and direct experience (mind) is biased,
restrained, constrained, limited,
enhanced, and facilitated in understandable and predictable ways by the physiological mechanisms
of human information processing.
Finally, this research effort is concerned with developing a “reference architecture” (a formalized conceptual framework for research and technology development) for designing physio-informatically robust interactive human-computer interface systems to the information-communication systems. The function of the reference architecture will be to provide insight into the various components of the system in the context of how they might affect the flow of information as it passes through them. The primary focus will be to consider the information flow between the human and the com-system in a sustained, iterative, experiential interaction. The intent of developing this reference architecture is to map the information flow during/caused by the intentional/volitional interaction with information between a conscious human and an info-com system.
For
several years our team worked to intelligently advance the physio-informatic
thesis of human-information interaction. Not only have our ideas been well
received and supported by both governmental and private institututions, but
also we have developed powerful core technologies, both software and hardware,
which will be the foundation of technical prototyping and development we would
do in this, the next stage of our work. The development of a fully functional grok-it
interface-system prototype will benefit those needing the types of
information interaction discussed here.
Dave
Warner MD PhD
Medical
Neuroscientist
Dir.
Medical Intelligence
MindTel
http://www.medibolt.com/documents/dave-warner-md-phd.htm
[1] Principles of Neural Science, 4th Edition, by Eric R. Kandel, James H. Schwartz, Thomas M. Jessell, McGraw-Hill, New York, 2000.
[2] Grok: to perceive and comprehend some sensory information or idea with a deep level of understanding (from the novel Stranger in a Strange Land, by Robert A. Heinlein, where it is a Martian word meaning literally ‘to drink’ and metaphorically ‘to be one with’)
