Welcome to The Mouse Sees, a project by emma rae bruml norton in collaboration with CultureHub on
the history of the mouse as a seeing object.
The Mouse Sees is meant to create discourse. Throughout the project I will publicly study and attempt to
locate a history of vision embedded within the computer mouse. The project will sustain, in its
materiality and performativity, a departure from Doug Engelbart’s notion of the user. Ultimately the
project and its interlocutors will convene online, at times asynchronously, at times in real time, for
the second Computer Mouse Conference, co-organized with Ashley Jane Lewis. The conference
will be held here on this page in the spring of 2021.
Through December 2020 I will be posting objects which have opened up questions around histories of vision
and the computer mouse. I will tweet when
there is an update in case you are interested in following along. Later in January I'll be livestreaming
The Dada of All Demos, a performance lecture which re-orients us to the conditions and tools
that made The Mother of All
Demos possible. It will be fun. I'll be channeling ~*~Douglas-Engelbart-Energy~*~ and
leading a collective scroll through a series of images. Here's a little history on the genesis of
In 1968 the Augmentation Research Center, Engelbart's lab, presented their work through a live
demonstration -- a kind of performance lecture. The presentation was centered on a new computing
contraption named “the oN-Line System”, or NLS. Although the NLS was the physical product of the
Augmentation Research Center’s work, Engelbart was always more interested in there even being a
possibility for exchange between the human and the computer. His commitment to the computer was a
commitment to thinking of intelligence as something to be augmented, not replaced. He had set out not
only to re-imagine what computers could do, but also to re-imagine what humans could do. His ultimate
goal was to produce a new kind of “knowledge worker.”1 The oN-Line System’s physical
construction included an amalgamation of devices, all situated within optimal distance for the human to
the machine: one hand to a mouse, another hand to a chord set and keyboard, and eyes to the screen.
The live demonstration of NLS, later coined “the mother of all demos”, performed some of the earliest
versions of hyperlinks, multiple windows, video conferencing, cross-computer collaboration, and perhaps
of the most importance, the use of a mouse. Windows, hyperlinks, and programmers at separate but
connected computers were all easily accessible to individual using the system. What set Engelbart’s demo
apart from other advances in computing at the time was that the oN-Line System kept the human embedded
within the mediation of inputs and outputs. As John Markoff notes, it kept “the man in the
loop.”2 Eight years after its invention in 1971 Bill English took the mouse out of its looped
system and into Xerox Parc. English’s original prototype was redeveloped for the first personal computer
the Xerox Alto. We can thank the Xerox Alto, a computer that heavily inspired the Steves of Silicon
Valley, for the desktop metaphor.3 The visual interface of files and folders and trashcans
are still with us today and through their tendency towards the visual they call for the use of a
1 Thierry Bardini, Bootstrapping: Douglas Engelbart, Coevolution, and the Origins
of Personal Computing (Stanford, Calif.: Stanford University Press, 2000), 107.
2 John Markoff, What the Dormouse Said: How the Sixties Counterculture Shaped the
Personal Computer Industry (New York: Viking, 2005), 355.
3 Here I am referring to Steve Jobs and Steve Wozniak who began Apple Computer
together in 1976. However, it’s worth noting that the canonical history of Silicon Valley, which we
are here to subvert, is studded with Steves: inventor of the digital camera Steve Sasson, creator of
the early video game Spacewar! Steve Russell, developer of early web mapping software Steve Putz,
inventor of an early version of the optical mouse Steve Kirsch, and wearable computing engineer
Steve Mann. The list does go on.
the mouse sees
Current object: the lightpen
last updated Sunday November 29,
One of my favorite histories of the screen, “The Random-Access Image: Memory and the History of the
Computer Screen,” is written by Jacob Gaboury. In the writing, we learn that a history of the screen is
dually a history of the vacuum tube.1 Early cathode ray tube screens saw a wide array of
input devices for pointing, tracking, choosing, and classifying. Because vacuum tube screens turned
information into light, they required an input device that shared the language of light as signal.
The mouse interrupted the history of the screen. It did not initially share with it a
common language of light.
The origins of light-bearing input devices belong to The Whirlwind
computer. Unlike the machines that came before it, the Whirlwind was able to display information in
real-time, a new moment for computers whose earlier capabilities were process-oriented and without the
use of a screen. The Whirlwind was able to receive inputs and produce outputs in real-time because it
used a three-dimensional array of magnetic core memory, an efficient organization of physical computing
components that allowed for the fast storing and retrieval of data. The team behind the Whirlwind
created an input device using the language of light, and this is where the mouse's predecessor, the
light pen, first materialized.
Operators of the Whirlwind could continuously interact with
information on the screen for the first time.2 Later, the ability to point out and interact
with objects on the screen became most useful in the Semi-Automatic Ground Environment. The SAGE
computer FSQ-7 allowed its operators to point out aircraft with a light gun.3 At this point,
the screen was still thought of as a substrate like paper, the analog substrate for tracking and
storing, and the light gun was the next best iteration of moving from the analog act of writing to the
digital act of pointing. In this sense, the problem of computing became not only the act of programming
switches but also the act of moving along x and y coordinates. The image above can be found on the
Internet Archive at this link.
1Jacob Gaboury, “The Random-Access Image: Memory and the History of the Computer
Screen,” Grey Room 70 (March 2018): 24–53.
2Charlie Gere, “Genealogy of the Computer Screen,” Visual Communication 5, no. 2 (June
3Paul N. Edwards, The Closed World: Computers and the Politics of Discourse in Cold
War America (Cambridge, Massachusetts: The MIT Press, 1996), 106.
In tracing a history of one forgotten device there appears another: the planimeter.1 Already
deeply familiar with the x-y space of the screen, Doug Engelbart credits his inspiration for the mouse
to the planimeter.2 The construction of the planimeter, invented in the early nineteenth
century, allowed for the calculation of an area of an irregular shape in two dimensional space. Years
later it was discovered that planimeters could be used in conjunction with one another to solve not only
integration functions but also differential equations.3 It wasn’t until Vannevar Bush used
torque amplifiers in his Differential Analyzer that this discovery could be physically performed through
computation.4 Pictured above is Jakob Ansler's 1856 planimeter. The image above can be found
on the Internet Archive at this
1 Marie-Jose Durand-Richard, “Planimeters and Integraphs in the 19th century. Before
the Differential Analyser,” Nuncius: Journal of the History of Science 25, no. 1 (2010),
2 Douglas Engelbart, “Stanford and the Silicon Valley,” interview by Judy Adams and
Henry Lowood, Oral History Interviews, March 4, 1987. https://purl.stanford.edu/cb260zz4895
3 Allan G. Bromley, “Analog computing devices”, in William Aspray, ed., Computing
before Computers (Iowa State University Press, 1990), 179.
4 Bromley, “Analog computing devices”, 180.