Welcome to The Mouse Sees, a project by emma rae bruml norton in collaboration with CultureHub on the history of the mouse as a seeing object.
The Mouse Sees is meant to expand discussion on the computer mouse. Throughout the project I will publicly study and attempt to locate a history of vision embedded within the computer mouse. The project will sustain, in its materiality and performativity, a departure from Doug Engelbart’s notion of the user. Ultimately the project and its interlocutors will convene online, at times asynchronously, at times in real time, for the second Computer Mouse Conference , co-organized with Ashley Jane Lewis. The conference will be held here on this page in the spring of 2021.
Through December 2020 I will be posting objects which have opened up questions around histories of vision and the computer mouse. I will tweet when there is an update in case you are interested in following along. On 2/24/21 I'll be livestreaming The Dada of All Demos, a performance lecture which re-orients us to the conditions and tools that made The Mother of All Demos possible. It will be fun. I'll be channeling ~*~Douglas-Engelbart-Energy~*~ and leading a collective scroll through a series of images.
One of my favorite histories of the screen, “The Random-Access Image: Memory and the History of the Computer Screen,” is written by Jacob Gaboury. In the writing, we learn that a history of the screen is dually a history of the vacuum tube. 1 Early cathode ray tube screens saw a wide array of input devices for pointing, tracking, choosing, and classifying. Because vacuum tube screens turned information into light, they required an input device that shared the language of light as signal. The mouse interrupted the history of the screen.
It did not initially share with it a common language of light.
The origins of light-bearing input devices belong to The Whirlwind computer. Unlike the machines that came before it, the Whirlwind was able to display information in real-time, a new moment for computers whose earlier capabilities were process-oriented and without the use of a screen. The Whirlwind was able to receive inputs and produce outputs in real-time because it used a three-dimensional array of magnetic core memory, an efficient organization of physical computing components that allowed for the fast storing and retrieval of data. The team behind the Whirlwind created an input device using the language of light, and this is where the mouse's predecessor, the light pen, first materialized.
Operators of the Whirlwind could continuously interact with information on the screen for the first time. 2 Later, the ability to point out and interact with objects on the screen became most useful in the Semi-Automatic Ground Environment. The SAGE computer FSQ-7 allowed its operators to point out aircraft with a light gun.3 At this point, the screen was still thought of as a substrate like paper, the analog substrate for tracking and storing, and the light gun was the next best iteration of moving from the analog act of writing to the digital act of pointing. In this sense, the problem of computing became not only the act of programming switches but also the act of moving along x and y coordinates. The image above can be found on the Internet Archive at this link.
In tracing a history of one forgotten device there appears another: the planimeter.1 Already deeply familiar with the x-y space of the screen, Doug Engelbart credits his inspiration for the mouse to the planimeter. 2 The construction of the planimeter, invented in the early nineteenth century, allowed for the calculation of an area of an irregular shape in two dimensional space. Years later it was discovered that planimeters could be used in conjunction with one another to solve not only integration functions but also differential equations.3 It wasn’t until Vannevar Bush used torque amplifiers in his Differential Analyzer that this discovery could be physically performed through computation. 4 Pictured above is Jakob Ansler's 1856 planimeter. The image above can be found on the Internet Archive at this link.
In 1968 the Augmentation Research Center, Engelbart's lab, presented their work through a live demonstration -- a kind of performance lecture. The presentation was centered on a new computing contraption named “the oN-Line System”, or NLS. Although the NLS was the physical product of the Augmentation Research Center’s work, Engelbart was always more interested in there even being a possibility for exchange between the human and the computer. His commitment to the computer was a commitment to thinking of intelligence as something to be augmented, not replaced. He had set out not only to re-imagine what computers could do, but also to re-imagine what humans could do. His ultimate goal was to produce a new kind of “knowledge worker.” 1 The oN-Line System’s physical construction included an amalgamation of devices, all situated within optimal distance for the human to the machine: one hand to a mouse, another hand to a chord set and keyboard, and eyes to the screen.
The live demonstration of NLS, later coined “the mother of all demos”, performed some of the earliest versions of hyperlinks, multiple windows, video conferencing, cross-computer collaboration, and perhaps of the most importance, the use of a mouse. Windows, hyperlinks, and programmers at separate but connected computers were all easily accessible to individual using the system. What set Engelbart’s demo apart from other advances in computing at the time was that the oN-Line System kept the human embedded within the mediation of inputs and outputs. As John Markoff notes, it kept “the man in the loop.” 2 Eight years after its invention in 1971 Bill English took the mouse out of its looped system and into Xerox Parc. English’s original prototype was redeveloped for the first personal computer the Xerox Alto. We can thank the Xerox Alto, a computer that heavily inspired the Steves of Silicon Valley, for the desktop metaphor.3 The visual interface of files and folders and trashcans are still with us today and through their tendency towards the visual they call for the use of a computer mouse.