Physio
Info Tronics for Perceptualization Environments
An
Anthrotronic Interface System To The Emerging Information-Communication
Matrix.
Advanced
research in the area of sensors and information systems continue to produce an ever emerging array of
networked information and communication (info-com) technologies. These in turn produce an ever
increasing volume of information
which must be perceived and understood by a human operator. We propose to develop an interface to the
network through which human communication of and interaction with this information is
facilitated. This paradigm
of interface technology is based on new theories of human-computer interaction, which are both
physiologically and cognitively oriented. This optimization of human interaction with
vast networked systems information assets incorporates multi-sense rendering technologies,
giving sustained perceptual
effects, and incorporates other natural user interface devices which measure multiple (physical and
physiological) parameters simultaneously and use them as inputs. Such a biologically
optimized interactive
information interface method has the potential to facilitate hyper-effective communication. This
increase in effectiveness will impact both human-computer and human-human communication through
extended perceptual
dimensionality and "enhanced expressivity.”
Introduction
The
amount and diversity of information being produced in the many fields of
human
knowledge grows rapidly. Whether it be academic researchers creating
and
publishing new information based on already existing material they have
utilized
in the development of new theories and methods, or the information that
is generated, used and recorded during more sudden and sporadic occurrences
of emergency medical response, for example, the volume and differences
in information increase. Complicating this state of affairs are the
integrations of information across boundaries and events made possible
by
emerging communications technologies. Cross pollinization and syntheses
of
the diverse data means that there are more and more possibilities for
action
when all this information is brought to bear on contexts of need. Interacting profitably with the many
sources of information using the many means of communication now available
requires some new models of how humans may interact with information
resources. In this text, our purpose is to articulate
one such model. Our research into human-computer interactions, particularly
as it developed conceptually with the needs of the disabled user,
allowed for some very effective ideas about how to optimize the human’s
ability to exploit the powers of information & communications systems.
Anthrotronic is a term we use to emphasize the human as central in the
design and use of information and communication technologies. The human
ability to perceive, process, and act on information is directly related
to the physiological structures and functions of the nervous system.
The
orientation of the model we propose is therefore towards a tayloring of
information
represenation to the unique capacities of the nervous sytem to take
in and respond to information.
History
and Problem
The
early stages of this research occurred in the neurology and rehabilitation
departments of a major medical school. All led to a new focus
on perception and expression as the key problems in re-thinking how
humans
might better interact with information systems. Historically, these
problems
were sidelined by the emphasis on faster more powerful computation and
greater storage. Human factors was, therefore, left out of the thinking
and
design of information systems. First we give a brief background on how
these
issues were formulated in the particular contexts of medical diagnostics
and rehabilitation. These were merely the starting point for the
interrogation of the logic of human-information interaction which we are
fully
engaged in at the present time.
Perception:
Electroencephalograms
(EEG) and Magnetic Resonance Imaging (MRI) devices were
both used to gather data. Electrophysiological output from the brain
is
what is thereby measured and output as squiggly lines (EEG) or crude
image
maps of the head (MRI). Experimentation with electrocardiograms (ECG)
was
also undertaken. This research led to the conclusion that the perceptual
forms of the data generated by these technologies was inadequate to
the task of providing precise diagnostic information to the clinician. Also, the form of the data locked out
anyone but the highly experienced clinician in making decisions over
obscure significance in the data. Based on this experience, the
perceptualization of data became a general problem to
be explored and addressed.
Expression:
Neuro-diagnostic
studies of the brain also triggered the awareness that electrophysiological
output comes not just from the brain, but from the muscles
of the entire body. The body is emanating certain kinds of energy as
measurable patterns at all times. EEG, EKG, and other physiological
monitors
are simply ‘reading’ this energy as data as it streams off the body.
The proximity of pediatric rehab to the neurology department occassioned
the question as to whether physiological output such as this could
in fact be exploited as an input source. That is, if the body is giving
off certain energies in certain patterns in relationship to some mental
or physical behaviors, then it is conceivable to use them to control
some
device like a computer. The output need only be captured in a way which
allows them to be converted into input for driving a device.
Based
on these insights the issues of perceptualization and expression within
information systems were raised and shown to be problems in need of
solution
for a range of users in many different fields. Rehabilitation and clinical/emergency
medicine have been the areas we have worked to penetrate with
both an articulation of the problem and a generic conceptual framework
for
its solution. More recently we have engaged with the humanitarian and
disaster
response communities to show the ways in which existent practices of
information gathering, representing and decision support are inadequate. Let it be noted that the only reason we
can make such claims is because technology has come into being that
allows wholly other ways of interacting with information. Our work has been to
research these technologies and within particular contexts to test and
refine our ideas about how to use them. The generic issues we now turn to.
Anthrotronic Principles of Human-Information Interaction
Anthrotronic
is human-centered and more accurately mind-centered thinking about
information and communications systems design. Rather than beginning
from
the perspective of power and storage, these are included in the larger
framework
of beginning from the needs and abilities of particular users as a ‘mind
in the system’ taking in and outputing information. Our assumption is
that
current interaction with vast and diverse information resources for
varying
and urgent purposes is hindered by
1)
A neglect of the multiple sensory systems of the human body, and
2)
Perceptually inferior preparations of information for those senses which
are
used.
The visual sense has been given top priority in the area of human-information
interaction. All our data is made available through the eyes,
with few exceptions. There are at least two other senses which are completely
cut out of the interaction with information: the ear and the skin which
has seldom, if ever been exploited as a means of information gathering.
Of course, the blind have benefitted greatly from it. Even
the way information has been prepared for the visual sense is often
inferior,
as it has been presented as text, numeric characters, and crude graphics.
In our research, the medical expert is always dealing either with printed
textual material, or else with the opaque visual forms of data produced
by medical instruments like the EEG, EKG, and such others. Text, numbers,
symbols, squiggly lines, and graphs have perceptual qualities which
are
inadequate to time and content requirements of those using the information
(eg. doctors, emergency medical personnel, etc.). The information,
as typcially represented, is just perceptually defficient, to the neglect of
the extraordinary capacity of our brain to capture and process information from
our senses. We are not saying that vision or text are
bad ways of accessing information. Rather, we are making a conceptual point
that while these work well for some needs, the critical nature of some
information
requirements is hindered by them. That is when great quantities and
diversities of information must be
accessed, represented in all their richness, and then used in a rapid
manner. Our goal is to offer conceptual (and eventually technological)
solutions for information demand by rethinking the sensory and perceptual
possibilities for how to render information to the human body for
decision support.
Solution
Our
goal is to take these concepts and begin to develop a system that
brings together different multi-sensory representations of information into one
unified dynamic interface.
The approach to the problems raised is based in part on ideas in the
study of sensory physiology. A
mind-centered orientation to human-information interaction asks first, “How
does the human nervous system through the senses gather raw data and then
present it as information to the mind?”
The answers to this question will help us create powerful interfaces
structures and functions between minds and data. Thus we are proposing to integrate the conscious human user into
this system as a computational resource: a mind (not just a user) in the
loop. By
increasing the number and variation of simultaneous sensory inputs, we
can
make the body an integral part of the information system, "a sensorial
combinetric
integrator". That is, the mind
and body inside the network interface we are talking about will be a locus of
percpetion and expression; a reader and a responder in any information and
decision intensive process.
To progress in this, we will identify the optimal perceptual
parameters in which information can best be rendered for each of the three
senses. That is, what types of information are best rendered to each
specific sense modality, optimization of represenation based on the unique
processing properties of the sense in question. Research
in human sensory physiology, specifically sensory transduction mechanisms,
shows us that there are designs in our nervous systems optimized for
feature extraction of spatially rendered data, temporally rendered data,
and
textures. Feature extraction is
defined by Kendal et al as “the selective detection and accentuation by sensory
neurons of certain features of a stimulus” (1991, 338). Models of information processing based on
the capacity of these neurophysiological structures to
process information will help our efforts to enhance perception of
complex relationships by integrating visual, binaural, and tactile sense
perception. Then by using the natural electrophysiological energy as a signal
source for input (referred to above), we can generate highly interactive
systems in which these biological signals initiate specific events.
Such a real-time analysis enables multi-modal feedback and closed-loop
interactions and, therefore, controllability and exploitability of the
multichannel,
multifunction concurrence of dynamically interconnectable bio-coupler
based hybrids (dave. Please translate from after the therefore in this last
sentence).
We will endeavor to address and solve the deficiencies in
information representation and decision support through both the
perceptualization of information and the enhancment of expressivity made
possible in a single interactive interface which can be deployed in any place
equipped to benefit from it.
Information will be rendered for three senses rather than one. The
eyes, ears, and skin will all become aveneus for gathering data, with vocal
an haptic signals allowing for command inputs.
Perceptualization of information is an idea which
assumes that under high intensity demand for diverse and voluminous information,
it is best to divide the different information up. After processing the different data through
data mining techniques, the goal is to take some of that data and put it into
a form which is highly accessible to the visual sense. Likewise, take some
of the data and make it accessible to hearing; and so with the tactile, or
haptic surfaces of the body. Large quantities of diverse data may be
transformed into multi-sensory forms of information. Here are the basic forms of perception and expression.
1) geometry, color, texture, and
dynamics becoming meaningful features of
information
which has been 'visualized'
2)
tone, pitch, timbre, volume, duration, location becoming meaningful
features
of information which has been 'sonified', or converted into sound
forms.
3)
touch, felt position, motion, and force becoming meaningful features of
information
which has been converted into forms which come into contact with
hands,
fingers, arms, or skin and muscle sensations of the body known as
tactile
or haptic manifestations. Think of the braille used by the blind,
and
imagine an ellaboration of this idea for accessing information without
sound
or image but with physical impressions across body surfaces.
Data
Mining & Knowledge Discovery
Data
mining processing is part of the core functionality of the interface. In
order to transform data into the different perceptionalizations
available to the user, an powerful
means of turning repositories of data into novel and powerful
information is required. "Gold mining is a
process for sifting through lots of ore to find valuable
nuggets. Data mining is a process for discovering patterns and trends
in large datasets to find useful decision making information … There
are
many different definitions of data mining. Almost all of them involve
using
the today's increased computing power and advanced analytical techniques
to discover useful relationships in large databases." "Data
mining
is a 'knowledge discovery process of extracting previously unknown,
actionable
information from very large databases'" [Aaron Zornes, The META
Group]
"Data mining is the process of discovering meaningful new correlations,
patterns and trends by sifting through large amounts of data stored
in repositories, using pattern recognition technologies as well as statistical
and mathematical techiques" [Gartner Group] quotes from www.spss.com.
It is important to note that data can come in many forms. Some indeed is in databases and data
warehouses. However, a good deal of the
data that a user of this interface will be interacting with is being generated
on the fly in crisis situations.
Medical monitoring at the site of a human emergency will be streaming
raw data into the grok-it interface for the user to perceive and respond to in
emergency time. Or if physicians are
doing clinical work over the web there will be simultaneous processing of
record data from databases as well as on the fly generation of patient data. Counteracting the effects of bioterrorism
would be another example of requiring data from both large stationary stores
and that coming from a vast array of different kinds of sensors at the and near
the site of emergency.
Objective
We propose to research, prototype and
evaluate an integrative interface matrix that couples the data streams
emerging from sensors, micro informatic technologies and databases
to the mind via an intelligent exploitation of the nervous system towards the
increasing of perceptual dimensionality and expressive capacity. This
Anthrotronic (human scale instrumentation system) interface matrix will allow
for the harnessing of the human nervous system in ways that increase
the user’s ability to “grok” (to perceive and comprehend
some sensory information or idea with a deep level of understanding)
and communicate the information being generated and transmitted
by the vast multi domain information-communications system. Further, we will
research, prototype and evaluate technologies that enable
Controllability and Exploitability
of the multi-channel, multifunction concurrence of dynamically
interconnectable bio-couplers to the info-com system. The foundation for this goal is the proposition that the
information flow between external sources (representation) and direct
experience (mind) is biased, restrained, constrained, limited, enhanced,
and facilitated in understandable and predictable ways by the physiological
mechanisms of human information processing.
Finally, this research effort is concerned with developing a
"reference architecture" (a formalized conceptual framework for
research and technology development) for designing Physio-informatically
robust interactive human computer interface systems to the
informations-communications systems. The purpose of the reference architecture
will be to provide insight into the various components of the system
in the context of how they might affect the flow of information as information
is passed through them. The primary focus will be to consider the
flow of information between the human and the com-system in a sustained,
iterative,
experiential interaction. The intent of developing this reference architecture
is to map the information flow during/caused by the intentional/volitional
interaction with information between a conscious human and an info-com
system.
For several years we have worked to intelligently advance the
physioinformatic thesis of human-information interaction. Not only have our ideas been well received
and supported by both governmental and private institututions, but we have
already developed some powerful core technologies, both software and hardware,
which would be utilized in any development we would do in the next stage of
this work. The opportunity to bring
into prototype a fully functioning grok-it interface system is to the benefit
of everyone with needs for the kind of information interaction we are here
envisioning.