THE GENESIS OF NASA RECON
Dr. William Mitchell
Professor of Information Science
University of Arkansas at Little Rock
NASA RECON is acknowledged by most historians to be the first multi-site on-line bibliographic system for diverse literature.
This paper examines the events that led NASA to contract with Bunker-Ramo Corporation for the development and test of a RECON
prototype in the mid-60s. The design considerations of the prototype are described and the events explaining the test conducted
in the fall of 1966 in four NASA research centers are chronicled. Even though the two-month trial proved the on-line bibliographic
retrieval concept and NASA desired to continue its deployment, Bunker-Ramo withdrew and NASA had to re-advertise and restart RECON
with another contractor. The reasons for these events are now revealed.
Charles Bourne and Trudi Bellardo Hahn deal with the development of on-line systems in their book (Bourne & Bellardo) much more
comprehensively than should be attempted here. This exposition focuses narrowly on the events and circumstances of three
turbulent years and deals with the disparate conceptions of the bibliographic retrieval systems that explain the diverse systems
that were realized in this period.
Design and Development of On-line Bibliographic Retrieval Systems, 1964-1966
Addressing the Third Annual Colloquium on Information Retrieval in Philadelphia in May of 1966, Rowena W. Swanson, then deputy
director of the Information Sciences Directorate within the Air Force Office of Scientific Research, discussed the engineering
aspects of building information systems. "The words ‘information’ and ‘system’ each contribute meaning to the term ‘information
system.’ Failure of information specialists to provide a definitional specification of systems concepts is all too prevalent
and may indicate lack of understanding of them. This understanding is second nature to systems and industrial engineers and
operations researchers”(Swanson, 1967, 3). Swanson enumerates examples of existing information systems for scientists, for
industry and commerce, for managers, for the military, and for libraries, highlighting approximately 30 of approximately 700
that she estimated to be documented in the United States in 1966. She then turns to survey the efforts to create national
networks, emphasizing published COSATI recommendations and the suggestions published in 1965 for a National Scientific and
Technical Information Library System. At the end of this survey she asks what all the fuss is about. In a system, she notes,
the whole is more than the sum of its parts, insofar as the parts, individually, are inert, needing each other to reach a
nascent state. How closely related are the poorly shaped information systems of the past two decades to repetitive abortive
phenomena? How many systems are planned, how many ad hoc? Is it the fault of the system or the environment in which it was
born or not permitted to grow that a system cannot respond meaningfully to its interrogators (p. 26)?
Swanson concludes her extensive paper (over 200 entries in her bibliography) with a summary of the evident directions of applied
and basic research in information systems that will be pursued by those she still considers to be pioneers of information
science. In making the transition from retrospection to future prospects, Swanson addresses the conflux of technology and the
professional library community. She does so by emphasizing the role that computers and systems technologists will play in
future information systems.
H. F. Mitchell looked at the future of the computer in early 1965. He foresaw a remote on-line facility costing less than
$100,000 (a $30,000 computer, a $1000 cathode-ray-tube console, and a $40,000 disk file) and time-sharing costing as little
as telephone service by 1970. If the computer is, as Mitchell characterized it, an adjunct to the main objective of information
processing and much of the equipment is on the shelf now, where are the information processing pioneers? Do information system
designers in science and technology, and possibly managers and investigating groups, subscribe to the belief, expounded in the
popular press, that automatic information handling depends on computers? The systems described in the foregoing sections
disclose a new pattern of cooperation between groups having particular information problems and computer people leading to
procedures that merge data base with equipment into a dynamic entity (pp. 26,27).
Not known to Swanson in May of 1966, Dr. Mitchell was but one of several non-librarian pioneers who were demonstrating on-line
scientific bibliographic information retrieval systems that year. These pioneers, for the most part, knew little of each
other. The currents of technology had by the middle 1960s simply made inevitable the evolution of on-line systems. What is of
interest is the different perspective that each of these pioneers brought to the task of information systems building.
Swanson enumerated the existence in 1965 of numerous off-line information systems that employed mainframe computers in batch
mode to search databases in response to queries. In terms of database size, the most significant of those focused on
bibliographic retrieval were operated by NASA, which began in 1962 running bibliographic citation searches on an IBM 1401, and
the National Medical Library, which made MEDLARS operational on a GE 235 in January 1964. Bruce R. Schatz, Librarian and
Director of the Digital Library Research Program at the University of Illinois at Urbana-Champaign, in his review of the
history of computer-based searching in 1997 for Science, characterized the early 1960s as the “grand vision” era and attributes
it to the over-optimism accompanying the birth of artificial intelligence.
The first attempts at realization of the grand visions, during the 1960s, centered around text search of technical citations.
The content was the text of a bibliographic citation of a journal article, which included the title, author, journal, and
keywords of the referenced article. A search query was matching specified words to words in the fields of the citation.
Prototypes of online systems for searching bibliographic databases were built in research laboratories in the early 1960s, and
the most successful of these evolved through the large-scale experimental testbed phase into commercial services (pp 328,329).
Schatz credits the Council on Library Resources studies, culminating in Licklider’s 1965 report Libraries of the Future, for
focusing the vision, but as Swanson had articulated, there were many spokespeople in the 1960s with a vision of a
computer-mediated information rich world. One such visionary was Simon Ramo, who published a 1960 talk he gave entitled “The
Scientific Extension of the Human Intellect” (Ramo, 1961). Dr. Ramo was a distinguished engineer and co-founder of
Ramo-Wooldridge, which had been responsible for systems analysis and management of the Air Force’s ICBM program in the previous
decade. He portrayed a future where communication and computer technology would support knowledge communities, allowing easy
on-line access to experts and cataloged expertise where all participants critiqued and augmented the community knowledge store.
Dr. Ramo believed that such a future was within reach of the technology that his firm was developing for the military. Swanson also
knew the nearness of such a vision of man-machine collaboration as she had funded Douglas C. Englebart at the Stanford Research
Institute (SRI) since 1962 in his Augmenting Human Intellect project. Englebart (1986, p. 75) remembers that the first annual report
under that contract “with the considerable help of Rowena Swanson, …was condensed into a chapter of a book published in 1963 [Vistas
in Information Handling, Howerton and Weeks (Editors), Spartan Books, Washington, D. C., 1963, pp. 1-29]”. Herbert F. Mitchell worked
for Simon Ramo.
In 1965 there were no on-line information systems that merited the designation “system”. Roger Summit had built a prototype
“conversational” inquiry machine based upon library catalog cards, called Converse I that he described in July 1965. He wrote
the data from 8000 catalog cards sequentially on magnetic tape, created inverted files on each of the significant title words
and citation descriptors, and then punched the records of the inverted entries to cards (38,285 of them) and stored the punched
descriptor cards in cabinets adjacent to a card reader and a teleprinter. A search was conducted by selecting descriptor cards
from the cabinets and constructing a deck that would isolate articles satisfying all descriptors (with the option of
interspersing NOT cards to change the sense of the descriptor).
Place the first inquiry card, with letters right-side up and facing the user, in the long card slot on the top of the machine.
Enter the inquiry card by depressing the metal bar on the back of the card slot. Repeat the procedure for each new card input.
When all the inquiry cards have been entered, insert an END card” (p 4).
After the END card was read and if three or fewer satisfied the query, the machine would print out the isolated catalog cards.
If 4-15 catalog cards satisfied the query only the card reference numbers would be printed. The user could then force the
printing of those catalog cards or add additional descriptor cards to narrow the search. If more than 15 cards satisfied the
query only the total number was printed and the user was expected to continue to reduce the number by specifying more
descriptors (hence the “conversation” would continue).
The Lockheed Electronic Sciences Laboratory in mid-1965 had plans for Converse II that would eliminate the card reader and
allow instead for the user to input terms on a teletypewriter. Supposing that the catalog cards for all 100,000 of Lockheed’s
in-house report collection were stored on tape, the user’s term would be matched against the descriptors and the response would
be the creation of a subfile of the article identifiers whose size would be output on the printer. The user could create any
number of such subfiles and would also have the capability of manipulating the subfiles by Boolean operations that would merge
and intersect the identifier lists, each time reporting the size of the resulting file. All intermediate subfiles would be
preserved to permit backtracking at any time during this interactive search. Converse III was foreseen to use a CRT for output.
In mid-1965 Lockheed knew only of experimental on-line bibliographic retrieval systems at MIT and at Harvard. In Summit’s view
Salton’s SMART system was a very interesting experiment, but it existed only in the lab and had no users. The TIP system at
MIT was already using teletypewriters (IBM 1050 communications terminals) for input and was accessible from over 100 such
terminals on the MIT campus starting in January 1965. The TIP system searched a collection of over 30,000 articles found in
several years of 25 Physics journals. Lockheed felt that the TIP system design was biased toward journal article retrieval and
would not work as well with a multi-disciplinary report collection (one of TIP’s unique features was retrieval of all articles
in the collection that cited the same sources as a selected article).
Paired with Swanson in the first workshop session, H. P. Burnaugh reviewed the BOLD system that he and Harold Borko developed
in 1965 as one of the projects of the Language Processing and Retrieval Staff at System Development Corporation. Although
conceived as an experiment and developed on a Q32 military computer with capabilities not then commercially available, BOLD
permitted a dozen workstations connected to the mainframe at SDC via the SDC Time-Sharing System to search a collection of
1745 abstracts from the Defense Documentation Center using any combination of 6883 retrieval terms (Borko & Burnaugh, 1966). The
BOLD user began a search by typing BEGIN/ and then entering any search term he choose followed by a question mark. The system
retrieved all similar terms in the dictionary and listed them along with the number of articles each referenced. The user would
repeat this process to learn the vocabulary of the collection and would then compose a query specification by typing phrases
that interspersed dictionary terms with AND, OR, and NOT. When the specification is complete, the user commanded SEARCH/ and
the system applied the query specifications to the data base and returned the number of citations satisfying all the
specifications as well as a table that showed each article reference ID and which of the dictionary terms it was associated
with. The user could delete rows from the table with a light pen and obtain any of the information on any of the articles,
including its abstract, by switching to browse mode. By commanding TYPE DISPLAY/ the contents of the CRT would be sent to the
The BOLD system made use of $100,000 CRTs and light pens as well as a computer comparable to an IBM 360/65. SRI and Englebart
had also used the Q32 two years earlier to demonstrate on-line information retrieval, as reported by Charles Bourne (1999):
"The first demonstration of an online bibliographic and full-text search system, and remote online searching, was a system developed
and demonstrated with Air Force and ARPA support at Stanford Research Institute (SRI) in 1963. This was done as an adjunct project
to the Augmented Human Intellect Program that was then underway at SRI under the direction of Doug Englebart. As Project Leader for
that online effort, and working with a very talented programmer, Len Chaitin, I tried to incorporate and make use of everything that
we had learned from the almost 20 operational batch search systems that had been demonstrated by that time. We felt that this online
approach was a natural and logical extension of what had already been done with the batch search systems. Our system made use of
programs and a database of full-text and bibliographic records that were prepared at SRI, and then installed and searched on the
Q-32 computer at SDC-Santa Monica from a small computer workstation (CDC-160A) 400 miles away at SRI."
In 1963 the Defense Department wasn’t interested in following up on the demonstration.
Borko, writing in May 1965, had not yet seen anything published about Converse I. “Prototypes of parts of such an automated
information center already exit at MIT, as part of Project MAC (Multi-Access Computer); at Harvard in SMART (Salton’s Magical
Automatic Retriever of Texts); and at System Development Corporation in the BOLD (Bibliographic On-Line Display) System” (p. 30).
Dr. Herbert F. Mitchell and the Direct Electronic Library
Dr. Mitchell received a Ph.D. in Applied Mathematics from Harvard University in 1948 where he and Grace Hopper worked under Howard
Aiken to build the Mark II. He joined the Eckert-Mauchly Computer Corporation in December of 1949, just before it was assumed by
Remington Rand, and headed the programming department for the first UNIVAC I. In 1953 the Univac Division was reorganized and John
E. Parker was made VP of computer sales with assistant Luther Harr. Dr. Mitchell was assigned to Parker as the senior technical
person for computer sales for business applications, responsible for sales demonstrations and more than 200 programmers who
programmed the demonstrations and did service bureau work. Dr. Mitchell held various sales support positions in the company after
it merged with Sperry, ending in 1960 as assistant director of product planning. In rapid succession he became Systems Sales Manger
for the Eastern US for Honeywell, then Honeywell’s Western Regional Sales Manger, and in 1961 was hired by Art Collins, founder of
Collins Radio Company, to participate in the design of a new switching computer, the C8401. A year later Parker, who in 1962
became the Chairman of the Board of the Teleregister Corporation in Stamford, Connecticut, invited Mitchell become the Vice President
for Advance Research starting in September 1962. Parker had already recruited Luther Harr as VP for Sales. In 1963 Parker became
President and CEO, so the old team was together again.
Telegister Corporation had been a subsidiary of Western Union until 1948 when it was spun off as a public corporation, but in 1962 it
still had less than 200 employees. It manufactured tote boards for the brokerage industry and got into the airline reservations
business with American Airlines in 1945, building a pilot seat inventory system that became operational in 1946 in Boston. The
prototype was followed in 1952 by a more comprehensive seat management system installed in the New York office. Teleregister built
the special purpose computer that ran the system, accessing a large drum, and used second generation computer technology so
successfully that eventually Teleregister handled seat reservations for TWA, United, Braniff, National, Northeast, Pan American,
Western, and several foreign carriers. Jon Eklund, Smithsonian Institution curator in the Division of computers, Information and
Society, concluded his 1994 review of this accomplishment with the words: "Reservisor was the first commercial system to combine
electronic processing and electronic communications. Because it ran so long and so reliably (the equipment operated 22 hours a day,
seven days a week, with a two-hour daily maintenance period), it provided a testbed and standard for large-scale heavy-duty
information-technology applications. Resevisor demonstrated not only that information could be reliably processed by a central
computer, but also that it could be processed at the time the transaction occurred or with negligible delay, and that the system
could process information with input stations scattered over a wide geographical area" (pp. 68,69).
IBM became a reservations competitor in 1958 when it contracted to develop the SABRE system for American Airlines (that became
operational in 1964), and in 1962 Univac was selected over Teleregister to handle reservations for Eastern Airlines. Teleregister,
it seemed, was viewed as too small to be depended upon for mission-critical applications. While struggling to stay in the airlines
reservation business by upgrading its customers from seat inventory systems to passenger reservations management, Teleregister
capitalized on its real-time communications expertise and its Telefile computer to implement three completely integrated large-scale,
on-line banking systems for the Howard Savings Institution, Union Dime Savings Bank and the Society for Savings, all installed in
1962. Teleregister also extended its dominance in the stock brokerage industry with the Telequote II system that put market data on
a broker’s desk. More than 1000 subscribers across the country were online in May of 1964. That year fourteen Telefile computers were
operational at six different sites (Weik, 1964).
Mitchell managed the research staff at Stamford and pursued ideas and products that would lead to new customers. In mid-1963 he worked
on a study funded by the National Security Agency that resulted in the specification of a new computer, the Teleregister Control
Processor, designed to effect information interchange among all the different computers at NSA’s computer installation at Fort Meade,
Maryland. NSA decided instead to move to IBM computers. In late 1964 he managed the development of an Automated Program/Budget Control
System for Air Force Headquarters. In early 1964 he worked on a proposal to update the United Air Lines Instamatic reservations
systems that Teleregister had installed in May 1961. In his autobiography (Mitchell, 1990) he recalled that:
"United wanted the computer system not only to handle the complete passenger reservation system (similar to what is now used by all
airlines), but also to track all the other operations within the air line — freight, flight schedules, pay, maintenance, profit and
loss, and even load and balance and fuel requirements for every flight. All of this for 1,500 flights per day, 250 airports, and 2,500
pilots. I had charge of the preparation of the technical portion of the proposal. In order to learn the details, United had me visit
their operation centers at New York City, Chicago, and several smaller centers. I had to learn how crews were scheduled (including all
union contract terms), how pay was calculated (different rates depending on whether the airplane was at the dock, on the taxi strip,
in the air over land, or in the air over water). I had to learn how priorities were handled for the loading of the plane — passengers,
freight, mail, special cargo, baggage, etc. Finally, I had to figure out how to get a computer to keep track of all these details in
real time, and do so in spite of individual component failures. It was a tremendously interesting assignment, and I was proud of the
result" (p. 146).
United’s management liked the proposal so much that it used it as a basis for an RFP to the industry in late 1964, and then in
December 1965 choose Univac instead of Teleregister to implement the new reservations system.
"John Parker sat on the Board of Directors of Martin-Marietta Corporation, and in early 1964 he learned that CEO George Bunker and
Bunker’s long-time friend Dr. Simon Ramo, who was then vice-chairman of the board of another Fortune-500 aerospace company, TRW, had
arranged to purchase a division of each company (financed by Martin-Marietta) and create the Bunker-Ramo Corporation “with the
objective of providing a unique corporation to fill what was seen as a national need in the application of electronics to information
handling” (Mitchell, 1965b, 5). Parker immediately suggested that Teleregister be added and the merger of Teleregister and Bunker-Ramo
was affected on July 8, 1964, creating a company with three divisions under CEO Simon Ramo and Chairman Parker: the Computer Division
of Thompson Ramo Wooldridge, Inc., the Electronics Systems and Products Division of the Martin Marietta Corporation, and the
Teleregister Corporation, with 3,100 employees and a combined annual sales of approximately $60 million. “It was believed at the
time of the formation of Bunker-Ramo that the electronics industry in this country particularly interested in computers, displays,
and other aspects of electronic information-handling largely emphasized hardware; industry failed to meet the requirement of equal
emphasis on systems engineering” (Mitchell, 1965b, 5-6). The new company sought to engage in certain specialized and key hardware
areas in electronic information handling and in the systems engineering of information handling.
Mitchell’s family resided in Los Angles and he had been working away from home for two years, so he immediately asked to be
transferred to the Canoga Park division. “Si Ramo spent a good deal of his time with the new company, and I became well acquainted
with him. My title was changed to staff vice president (whatever that means), and I was given the mission of developing the
commercial side of the division’s business, which had become almost exclusively Federal-Government oriented” (Mitchell, 1990, 148).
Dr. Ramo was acquainted with Melvin Day, then director of the Technical Information Division at NASA, and was aware of Day’s interest
in developing an on-line bibliographic retrieval system at NASA (more below). Ramo and Mitchell talked about the similarity of batch
bibliographic retrieval, the existing scientific currency awareness services, and airline reservation retrieval and concluded that
Mitchell should draw up a proposal to NASA. Mitchell completed this task in the early fall of 1964 and the 14-page unsolicited
proposal for a Direct Electronic Library (Mitchell, 1964) was immediately delivered to Melvin Day.
The emphasis in the report was on the cost of setting up a nationwide network to connect 15 NASA installations employing 34,236
personnel, one-third of which might make use of the library, and thus requiring the distribution of 102 CRT consoles among the
installations. In a single center configuration these CRT consoles would communicate with a single computer and its disk file
controller, which limited to 100 the number of concurrent searches that could be accomplished across the system. By increasing the
number of centers in a network the proposal showed both that the number of parallel searches could be increased as well as the
volume of citations stored on-line, but anticipated a corresponding increase in demand for access. A single center system was
projected to cost $54,500 per month and provide for 54,500 searches per month to NASA at the targeted $1 per search, with a peak
capacity of 350,000 searches. A three-center system was estimated to cost $108,500 per month, hence providing 108,500 searches at
$1 each, but having a peak capacity of 1,050,000 searches per month. If 12 centers were configured, the cost grew to $349,000 per
month with a peak capacity of 4,200,000 searches (over 5 searches per workday for every NASA employee).
The search strategy was described in a single page, first defining a direct reference, the list of accession numbers returned by
specifying a single coordinate value (author, sponsor, etc, perhaps within a date range), and then indirect references, a list of
accession numbers generated by user-chosen key words.
The computer at the center will attempt to match each such key word against a master list containing all indexed key words for the
chosen category plus all anticipated synonyms of these. As each successive key word is entered, the computer responds with a
rejection indication if it fails to identify the key word, or the number of articles which are associated with that key word (or
its synonym) and all earlier accepted key words. Further restrictions upon selection may be imposed by specifying inclusive dates
of publication, role status of the key word, type of article, size of article, sophistication of treatment, etc., as is deemed
desirable. Each specification adds to cost of indexing, of storage, and of selection. When the number of isolated articles has
reached a sufficiently small value, the user requests display of titles, and then proceeds as described for a direct reference
(display of the full citation and instructions for ordering the full article] (p. 5).
To Ramo’s and Mitchell’s knowledge in 1964, no on-line searching was being done, but they saw no technical impediment to constructing
a large-scale system for NASA that would permit coordinate searching of bibliographic citations by over 1000 simultaneous users
nationwide with reasonable response times.
Bibliographic Information at the NASA Scientific and Technical Information Facility
NASA initiated a program to disseminate bibliographic information about the aerospace literature in 1962 by contracting with
Documentation Incorporated to abstract, index and publish an abstract journal entitled Technical Publications Announcements that was
renamed Scientific and Technical Aerospace Reports (STAR) in 1963. STAR focused on report literature generated by or of interest to
NASA and was complemented by a similar NASA-supported service provided by International Aerospace Abstracts (IAA) that covered the
journals, books and other “open literature” of aerospace. “The two journals together attempt to provide comprehensive access to the
world’s current literature dealing with aerospace science and technology. By special arrangement, STAR and IAA are issued in
coordination with each other and on alternate weeks. Both use identical subject categories to group their announcements, and both
contain basically the same indices” (Brandhorst & Eckert, 1966, 3). From January 1962 through December 1965 the bibliographic
citations converted to magnetic tape by the Scientific and Technical Information Facility (STI) grew to well over 200,000. Batch
searches using inverted files were performed on this collection using an IBM 1401 until the Linear File Search System became fully
operational on an IBM 1410 in December 1964. NASA offered the tapes and several organizations accepted, producing their own programs
to manipulate the tape data according to the needs of their own particular operating environment. Most of these operations involve
severely re-formatting the NASA file with considerable loss of the non-subject data the Facility finds necessary. They make no attempt
to be the equivalent of the Facility’s present 1410 system, being contributions to the art in their own right. Three of these efforts,
each involving IBM-7090 series equipment, at the University of Pittsburgh, Republic Aviation, and North American Aviation, are
described in References 12, 17, 20-22. Additional activity, involving a CDC computer, is currently going on at the University of
Indiana [the references are dated from December 1964 to December 1965] (Brandhorst & Eckert, 1966, 14).
None of the previously mentioned pioneer on-line systems apparently were involved with NASA’s efforts.
A batch searching service was offered by STI and Brandhorst reported in 1966 that it “has prepared, edited, and delivered over 1000
searches in 1965, all of which, if not exclusively machine searches, received their major contributions from a machine search” (p. 34).
Imagine Day’s excitement at Bunker-Ramo’s suggestion that over 50,000 searches per month could be conducted from any NASA facility in the country at a cost of $1 each!
Van A. Wente directed systems development at STI and plotted the strategy for introducing on-line retrieval (called the Retrieval
Dialog Study (Simpson & Flanagan, 1966, 318), this could have been the motivation for Roger Summit to rename Converse II). He notes
NASA had already placed in many field center libraries duplicate copies of the documents indexed at STI. “The documents, in nearly
every case, could be viewed immediately either as journal articles, books, proceedings, full-size reports, or microfiche back to
1962” (Wente, 1971, 97). These duplicate libraries were under-utilized because of the effort involved in manual searching the paper
indices, so the missing piece was on-line citation searching, which NASA had already decided to call RECON, short for REmote CONsole
(Bourne & Bellardo, in-press). Dissemination of access to the citation collection at STI, precisely as Mitchell’s proposal described,
would permit in most cases not simply the identification of a document that could be delivered in several days, but the alerting of
a local librarian who could supply the document within the hour! “Indeed, any failure to have full document text available would
clearly break the full chain of feedback and iteration which on-line systems usually employ” (Wente, 1971, 98).
Bourne reports that during the fall of 1964 Roger Summit had occasion to discuss his ideas about Converse with Melvin Day and he
followed up with an unsolicited proposal to NASA to extend Converse to the NASA collection (Bourne & Bellardo, in-press). Summit,
like Borko at SDC, Kessler at MIT, and many others in the early 1960s were working in experimental time-sharing environments. In
contrast, Mitchell was proposing a real-time dedicated application derived from years of experience in seat reservations, stock
market quotations, and on-line banking, and employing existing commercial technology. NASA knew that there were still other
organizations with interest and technology appropriate to mounting a bibliographic retrieval system of this scale so Wente developed
specifications that in April 1965 resulted in NASA’s issuance of a request for proposals to develop a prototype system employing the
full NASA collection, then about 200,000 documents, in a realistic environment of research libraries with direct use of the system
by working NASA scientists and engineers. By a formal competitive process, NASA selected the Bunker-Ramo Corporation to conduct the
test principally at three NASA locations using remote terminals and programs owned by that company and operated through a UNIVAC
1050 computer located in New York City (Wente, 1971, 96).
Day apprised Ramo of the decision to issue an RFP so Mitchell began work on the winning proposal in February of 1965. He continued
to call the system the Direct Electronic Library because NASA had not shared its chosen name. He published his 18-page prototype
proposal dated April 12, and distributed it to various company officials ten days later, including Si Ramo on April 23, 1965
(Mitchell, 1965a). During those ten days he developed the algorithms that demonstrated that the features described could be
performed. In contrast to the previous emphasis on systems level communication and cost, the prototype devotes all but three pages
to describing the user’s search interface as was requested by NASA’s RFP.
Mitchell’s charge was to find commercial applications for the expertise of the Canoga Park Division. He had Ramo’s support that there
was a commercial future to on-line information retrieval and his idea was to develop a system that would permit users to select any
of a variety of information databases, combining all of the existing medical, legal, and scientific information services into one
system. "NASA’s RFP seemed to me to be an ideal way of getting started in the broad field I had been proposing, and I persuaded our
management to bid on the contract. I also persuaded management to do the programming without charge so we could have proprietary
rights to it when the contract was completed. Otherwise the government would own those rights. Of course, we won the contract, as
our price was so much lower than the other bidders, and we jumped in to get the job done" (Mitchell, 1990, 149).
In the prototype proposal the 40 lb Bunker-Ramo 213 CRT terminal is described as the user console with its row of 16 function keys
re-labeled to invoke search actions (however, the picture shows a terminal with function keys labeled for an airline reservation
application). The user begins a search by selecting the collection of citations to be searched.
For the average user it is anticipated that several or many collections will be available through his console, some perhaps requiring
proper identification on his part before access will be granted by the System. The user initiates his reference by depressing the
“Collection” function key, followed by the keys describing the desired collection, and terminated by depressing the “Transmit” key”
(Mitchell, 1965a, 7).
If the entry was not recognized, the response is a numbered display of all of the available collections, whence the user chooses a
displayed collection by pressing the “Collection” function key followed by the number of the list entry, followed by “Transmit,” or
possibly continues to the next page of the list by simply pressing the “Collection” function key.
Once the collection is established the user can isolate relevant citations by using the “Report number” function key, the “Accession
number” function key, the “Author” function key, the “Sponsor” function key, the “Keyword” function key or the “Date Range” function
key. In all cases, should the entry following the function key not be recognized by the System, a numbered list of acceptable entries
is presented, and if the entry is recognized, it is added to the specifiers already accepted and a numbered list is displayed
showing each accepted specifier and how many citations in the collection remain when this last specifier is imposed in addition to
the previous ones. Keyword entries are sometimes subject headings and when this occurs the response is a numbered list of the terms
subsumed by this heading. If the entered keyword is not recognized a list of index terms with the same stem prefix will be returned.
Provision was also made to “or” into the working set citations isolated by additional author, sponsor, or keyword specifiers by means
of inserting a “/” between the function key and the specifier (whence, if recognized by the system, it will be listed as an
ALTERNATIVE keyword and the size of the working set would usually increase with its addition). Provision was also made to erase any
specifier in the list of specifiers, nullifying its effect. “The erase feature is particularly useful for examining several sets of
titles based on a series of specification sets, of which one term only is varied. This process may be repeated as often as desired”
(p. 13). At any time the user may press the “Title” function key and see a numbered listing of the full citations in the current
working set (each one possibly requiring multiple screens). The “Page” function key is used to view the next screen, or, followed by
a number, to jump forward or back to the designated citation entry. Should the user find articles he wants to see, he presses the
“Deliver To” function key and follows with an entry identifying the delivery information. If the “Deliver To” function key is pressed
when a subsequent citation is being displayed, it can be followed immediately with the “Transmit” key and the same delivery
information is assumed. Pressing either the “Terminate” or the “Collection” function keys will reset the terminal for a new search.
In addition to citing the 100 concurrent user limitation in several places (the single center configuration) the prototype proposal
cites a maximum response delay of three seconds when the line load is less than 70% of capacity, and promises that the planned
installations will average only 50% line loadings. It also enumerates ten statistics that the System will compile, including hardware
usage and failures, frequency distributions of keywords employed by user, referenced and un-referenced index terms, number of
specifiers per search, citations referenced and un-referenced, and new keywords and accessions entered per time interval.
This document was given to the Eastern Technical Center (the Teleregister Division) to refine as Bunker-Ramo’s response to NASA’s
RFP. No doubt the proposal and contracts people in Stamford included some of the same verbiage they had included in the United
Proposal in April of 1965, such as "The Bunker-Ramo Corporation considers the development, design, implementation and operation of
on-line, real-time systems its primary business and had specific success in managing, developing, manufacturing and operating
extensive large-scale, on-line systems operating in real-time. The Bunker-Ramo Corporation has had more experience than any other
corporation in the installation and maintenance of large-scale on-line systems in business and industrial applications (An Electronic
Data Processing System, 1965, 2,4).
References to the past accomplishments of both TRW and Teleregister together with a proposed price of $235,000 for the large-scale
demonstration offset the fact that the Direct Electronic Library existed only in Dr. Herbert F. Mitchell’s imagination. NASA was in
the position of selecting from among proposals from organizations with working on-line retrieval prototypes but no experience with
large-scale, real-time applications, and a proposal from a company that had an excellent reputation for delivering exactly this
kind of system but which had not built a prototype. NASA awarded contract NASw 1369 to Bunker-Ramo in June of 1965.
The Bunker-Ramo Version of RECON
Milton Mohr, the head of the Computer Division in Canoga Park, was not pleased with the contract. The work was to be done by
Teleregister Division people (Dr. Mitchell being one), on Teleregister equipment, and it was under-priced with the difference to
come out of his budget. He refused to spend any of his R&D budget on the project and left Mitchell to scramble to piece together
the necessary resources. Dr. Mitchell teamed with Dr. Dale Scarbrough, an electrical engineer who came with Simon Ramo from Hughes
Aircraft when Ramo-Wooldridge was formed in 1953 and who now held the title of Senior Member of Technical Staff of Bunker-Ramo, to
get the contract done. Mitchell recalls that Dr. Ramo was not so interested in the NASA contract as to protect Mitchell’s time.
"In 1964, Dr. Ramo had been given the Air Force’s award as the man who had made the greatest contribution over the first ten years of
the Air Force’s guided missile program. This gave him considerable prestige in space circles, and he apparently decided to make use
of it in September 1965. He had me present a proposal to the Goddard Space Flight Center for Bunker-Ramo to provide computer
expertise to the space center as it installed its $25,000,000 order from IBM of their latest and fastest model 360 computers…I and
my colleague in the information retrieval contract (Dr. Dale Scarbrough) were proposed as the super-experts. I learned later that
Dr. Ramo had told Dr. Clark, Goddard’s director, that I was one of the most knowledgeable people in the business! Dr. Clark bought
the idea, but before it could be implemented he was transferred to NASA Headquarters in Washington, and the temporary director was
persuaded by the operating management that such an arrangement would be too difficult for them to handle" (Mitchell, 1990, 149).
In the Appendix of the proposal (Mitchell, 1965b) Mitchell’s vita identifies him as staff vice president “responsible for the
development of new applications in the areas of real-time systems and information retrieval. Developed a system for remotely
searching a large technical literature index using subject specification terms, known as the Direct Electronic Library” (p. A1).
Scarbrough’s vita says “currently active in systems design of nation-wide remote interrogation of complex technical information
data banks” (p. A4). The proposal was time critical and promised “each of these individuals would spend over the next six months
a considerable fraction of his total time on the Goddard problem. Drs. Mitchell and Scarbrough would call on other specialists
largely at Canoga Park to furnish specific analyses, judgments,...” (p. 9).
Mitchell devoted as much effort as he could spare to the NASA contract over the next year but it wasn’t his only responsibility. He
contacted Mel Day and made several visits to the data processing center at Suitland, MD to arrange for the magnetic tapes containing
the 200,000 citations and documentation as to how to decipher them. This was a non-trivial task because NASA had never attempted to
deliver the entire database to a non-classified organization. STI had to write a program to identify all of the classified documents
on their tapes and overwrite those entries with spaces. They then they had to feed their tape library through the program. The IBM
1410 was a character machine and the citation records were each variable length in three segments delimited by reserved characters.
Mitchell also arranged to acquire part of the effort of three programmers from the group he supervised the previous year on the Air
Force Budget project to begin the design of the Univac 1050 programs. In Canoga Park Mohr permitted the project to use the GE 425
computer that did the division’s business data processing on the third shift. A programmer was found and set to working on a program
that would read STI’s 1410 tapes on the 24-bit word GE machine and output inverted files for the Univac 1050, a 36-bit word machine.
This task was further complicated by the fact that the GE assembler was not well documented and the tapes were not formatted reliably
(NASA contributed extra funds to compensate for the over two months of delay the errors caused).
During the last six months of 1965 Mitchell and Scarbrough flowcharted the system and discussed the search interface occasionally
with colleagues. They viewed the job as the straightforward identification of a citation satisfying criteria and they knew how to
efficiently manipulate inverted files. The search strategy was simply a logical interaction to input criteria values, so they needed
little help in creating an efficient interface (in style like a reservations interface). Their goal was to permit the isolation
parameters to be input as quickly and error-free as possible.
Other factors influenced the design of the search interface so that it evolved from that described in the proposal. The console was
upgraded to a Bunker-Ramo 203 Display Station that had a larger screen and a detachable keyboard but fewer keys. As a consequence
of learning more about NASA’s work environment, the search procedure now began with the user identifying himself by depressing the
“User” function key and following it with one or more lines of identification terminated by the “Transmit” key. The System (now
called RECON) responded with a numbered list of the series that the user was authorized to search (such as STAR N-10,000) and the
user selected one or more by depressing the “List” function key, following it by list number or numbers separated by commas, and
ending with “Transmit.” Input of lists was a new feature, as was avoidance of error in specifying the citation collection via
selection from a user-specific list.
RECON then displayed the status of the “login” by echoing the user’s identification and chosen accession series and was prepared
to accept specification terms: Author, Corporate Sponsor, Report or contract #, or Subject keyword. As before, an entry that did
not match a term in the index list provoked a numbered “near” list of the valid terms, but now a valid entry also provoked a near
list unless the function key had been pressed twice before entering the term. The rationale was to show more possible terms of
interest that could be included in the specification or to allow the user to substitute a near term for the original one. When the
near list is displayed, the user had to select one or more position numbers (the list may be scrolled up or down with the + and –
keys) by again pressing the specifier function key followed by the position numbers selected (ending every input with “Transmit”).
In the event that subject keywords were being selected and one or more happen to be cross-referenced, RECON would interrupt itself
and display a new list showing each of the keywords just chosen, each followed by the number of citations it referenced, and then,
appended to this listing, a listing of all the cross-referenced terms, numbering from 51. The user made another selection from this
augmented list by following the “Subject” key with the number(s) of the desired term(s) (original or cross-referenced), and, if no
more cross-referenced terms were selected, RECON echoed the status of the selection criteria by incrementally adding the selected
terms to the Search Status display and permitting the user to confirm or exclude its application.
In the Search Status listing the terms selected were displayed in a numbered list, each followed by the number of citations so far
isolated in the working set. The user pressed "+" followed by the specifier key to accept the last displayed term, and if so, the
next term was displayed with the number of citations that would result when its references were “or’d” into the working set. If the
user rejected a term by pressing "–" followed by the specifier function key, that term was dropped and was replaced by the following
term in the list with the process continuing until the list of terms chosen by the specifier function key was exhausted (END was
displayed by RECON on the last line of the Search Status listing). The user could then “and” additional specifiers by pressing
another specifier key and entering its value, could enter an “and not” specifier by first pressing "–" and then the specifier
function key followed by a value, or could “or” in alternate references by pressing "+" and then the specifier function key
followed by a value. The number shown after the last numbered entry in the Search Status screen was the size of the current working
set and the user was always given the opportunity to see how the size would change before confirming the application of a new
specifier. If the user reviewed the search status listing and wished to delete any specifier, he entered "-", followed by the list
number of the specifier and “Transmit” and the search status was updated to show that the term had been removed and the new size
of the working set as determined by the remaining specifiers.
When the user wanted to see the citations isolated in the working set he pressed the “Title” function key and “Transmit.” The
citations in the working set were be displayed beginning with the most recently accessed and continuing through to the oldest, and
the user could jump down the list by pressing “Title” repeatedly, or order the current citation by pressing the “Order” function
key. When the user was finished with the working set, he pressed the period key (.) and “Transmit.”
Two other functions were provided: “Message” to dump the search status to the default librarian’s printer (the only provision made
for printing the CRT display), and “Instruction” to access a tutorial, or, if followed by a second function key, to receive help
about the use of that key (without affecting the state of the search). There was also a date key that accepted a range parameter and
limited the working set to citations that satisfied that range. Date restriction, specifiers entered in the suppression mode (-),
“ANDed” specifiers, and “ORed” specifiers were all applied to the whole working set.
Mitchell completed the conversion of the initial 200,000 citations (filling over 50 magnetic tape reels) in June of 1966 and
personally delivered them and a users manual dated May 2, 1966 (Mitchell, 1966) to the Teleregister teleprocessing center in New
York City. The manual described from the user’s perspective what the flowcharts had specified that the program should do. Mitchell
continued to devote himself to other tasks. For the next five months STI sent update reels every two weeks to Canoga Park to be
processed and then shipped to New York City so that the scheduled demonstration would provide access to the most current information.
On the east coast the programming team had been working from the flowcharts and had coded about 60 modules that had to be debugged
now that the data was available. However, the Univac 1050 was providing services to Teleregister Division customers during the
day so the programmers could only access the machine at night and on weekends. Site preparations were begun so that 23 consoles could
be installed in six NASA centers, eight at Langley and six at NASA headquarters. NASA provided idle telephone circuits that were
serviced by Western Union and a seven-week trial began in October 1966. Searching was supported all day long in the foreground
partition of the Univac 1050 resulting in 800 or more searches a week, but bad telephone connections precluded the more distant
sites from communicating with New York City. Bunker-Ramo had arranged for David Meister and Dennis Sullivan from their human-factors
group in Canoga Park to conduct a formal evaluation of the trial and their report was both delivered to NASA (Meister, 1967) and also
published (Sullivan, 1967). Both had assisted Mitchell with refining the design of the RECON user interface and they had made the
rounds to train and interview librarians and NASA engineers at all of the sites. Because 40% of the searches occurred at Langley,
most of the systems evaluation was based on data that they collected at that site.
However, the tested system was not quite the same system that Mitchell had described. The users manual for the test (Meister, 1967,
Appendix D) shows that several modifications that were made after Mitchell delivered it, some merely editorial and some substantive
(these documents were hand typed). On page 7 (and pages 9 and 22) it is noted that the “AND NOT” (suppression via "–", “Subject”,
term entry) was not available. On page 11 (and page 23) deletion of previously applied specifiers is restricted to the last applied
specifier. On page 17 the bypass option (selecting after pressing the function key twice) is not available when selecting from a
near list. On page 20 the feature of invoking the current search status is added (previously the Search Status was a System response
to a change in the working set specification). On page 26 the “Message” action is missing. On page 27 the specifications that error
messages would include reference to the tutorial are deleted as is all mention of a busy message that was to appear, prohibiting the
initiation of a search when the system was overloaded. Apparently the programming team had encountered problems coding all of the
As the testing proceeded, users discovered that RECON did not behave as presented in their manual (actually, the manual was
ambiguous). The evaluators note that “when the OR function was employed, the term which had been ORed was locked to the immediately
preceding term in that logical statement and could not be dissociated from it” (Meister, 1967, 14). In describing the step of entering
an alternative specifier (prefixing a specifier function key with "+") on page 11 the manual says “It may be desirable to enter a
specification term on an either/or basis with the previous specification term [emphasis added]. As we have seen, it is convenient
to do this by multiple selection from an alphabetical or cross-reference list, but this is not always possible”. When demonstrating
this function on page 21 the manual notes that the illustration shows an increase in the size of the working set. “This is because
the new term was “ORed” to the preceding terms [emphasis added], which means that the search for documents referencing MICROCIRCUIT
is cumulative with the preceding terms [emphasis added].” Had the delete functionality been implemented as described, there would
be no question that ORing was cumulative since it would have been possible to delete the second to the last specifier in the search
status (invalidating the “lock”).
Both the limited delete and the forced association of the OR were the focus of user criticism about the logic of the system. Users
complained that A and B or C was always interpreted as A and (B or C), requiring (A and B) or C to be entered as (A or C) and (B
or C). If A were the keyword “space” and B and C were authors, and the user wanted to specify articles on space written by either
of two authors, the system gives the expected result by entering “Subject”, space, “Author”, Smith, "+", “Author”, Jones. But the
designer intended this sequence of entries to isolate the articles Smith wrote about space together with all articles written by
Jones, so that the designer would now continue with “Subject”, space to accomplish the goal of limiting the set to articles about
space (the need to repeat the specifier indicates that the designer envisioned that a query involving either of two authors and one
subject would be entered authors first, observing the pattern “expand then narrow”).
When the user wanted either articles on space written by Smith or articles on mars, the sequence “Subject”, space, “Author”, Smith,
"+", “Subject”, mars isolated the expected articles by Smith but only those articles on mars that happened also to be indexed under
space. The designer had intended the sequence to produce what the user wanted, but lack of communication, lack of quality programming
time, or the inadequacies of the Univac 1050 resulted in a flawed implementation of Mitchell’s design. Note that the user should not
begin with the references to mars and expect to OR in the articles by Smith on space because the AND implementation is never right
associated (“Subject”, mars, "+", “Author”, Smith, “Subject”, space, "+",”Subject”, mars is required to isolate the desired set).
Any design based upon incrementally manipulating a single working set will be restricted in the Boolean expressions it can generate
and therefore will be more cumbersome to use than a system, such as Summit had designed, where multiple working sets are manipulated.
The second area of criticism highlighted in the evaluation concerned system response to the “Transmit” key. In his 1964 proposal
Mitchell had predicted that searches would average ten question-answer sequences and require only three minutes of elapsed time to
complete. Given light line loadings, he envisioned response times less than 3 seconds and service intervals about fifteen seconds
(average time between presses of the “Transmit” key). However, he had assumed a dedicated computer that was limited by the speed of
its disk controller (two disk accesses of 11 milliseconds each to answer each transmission).
The Univac 1050 became available in 1963 and was intended as “a general purpose subsystem employed mainly to supplement the parallel
processing capabilities of UNIVAC III, UNIVAC 490 Real-Time and UNIVAC 1107 Thin Film Memory Computing systems” (Weik, 1964). The
machine supporting the test had 16K of 4.5 microsecond RAM (United Airlines had been quoted a GE 635 with 1 microsecond memory six
months earlier) and was attached to a Fastrand II Drum Storage System that held 132 million characters of data with an average access
time of 92 milliseconds. The computer ran two partitions, foreground and background, so NASA shared cycles and memory with a program
doing processing for a New York brokerage firm. This was clearly not the machine that Mitchell had envisioned to service the Direct
Electronic Library, but the Teleregister Division had no other computer available and there was no funding to rent time on a faster
computer. But neither were 100 concurrent searches being conducted. However, the estimates Mitchell had made as to disk accesses
required to support a search were now low by an order of magnitude because the program was running off the drum, and the drum itself
was slower by an order of magnitude, so the expected response time of below 3 seconds for 100 concurrent users became the expected
response time for a single user.
Mitchell had initially anticipated that even if users knew exactly how to conduct their search they would spend most of the search
duration keying input into their terminals. With the change in the search interface to minimize keying by presenting lists to users,
the dialog between user and computer became heavily dependent on the computer’s ability to compute and transmit characters. He had
foreseen that the heaviest load on the computer would be when users requested title display and so the user manual cautions “It is
therefore recommend that you try to narrow your search terms as precisely as possible, to the point at which only a reasonable
number of citations will be displayed for your viewing. This will be a convenience to you and a courtesy to others who may wish to
use the RECON console” (Mitchell, 1966, 10). The evaluators made viewing a citation the criteria of a successful search and published
the count of citations viewed, reporting that the average successful searcher pressed the “Title” key 9.3 times (only 39% of the
attempted searches pressed the “Title” key).
Because of limited memory the Univac 1050 programs kept all user data on the drum, which meant that very few requests could be
responded to in less than a second and that extensive inverted file manipulations involving creating a new working set on the drum
would have taken several seconds. Under these circumstances the average response time was likely close to 3 seconds. If a search and
display of results averaged 40 “Transmit” presses (due to increased interaction with lists), each averaging 3 seconds of response
time, and if all users took an average of a 10 seconds to compose the next transmission (the terminal was locked until the computer
responded), the average search would be about 9 minutes (this was the typical search time later reported by Summit (Lancaster, 1973,
192-196)). During the 10 second lull the computer could generate responses to 3.3 other users. Since the load on the system averaged
about 180 searches a day during the trial, there would have been an average of 3.4 concurrent searches if the search distribution
had been uniform, which it was not. It is likely that often during the trial there were more than 10 concurrent searches going on
and it is not surprising that with likely ten to fifteen users sharing the processor the evaluators were able to measure mean delays
not of 3 seconds, but of 38.5, 133.7 and 74.5 seconds on different searches (Meister, 1967, 15). It should have come as no surprise
that as soon as more than 3 concurrent searches are attempted on this computer system, the response time would have begun to
deteriorate. Since more than three concurrent searches was routine, many of the “failed” attempts and perhaps a good portion of the
number of initiated searches can be attributed to bad-response-time frustration and repeated retries.
The third area of complaint was directed not at RECON but at what the users considered to be the poor quality of indexing. “RECON
users were brought into very close personal contact with the characteristics of the present indexing system; this intimacy caused
them to attribute some of their dissatisfaction with indexing terms to RECON” (Meister, 1967, 15). The evaluators analyzed system
usage satisfaction separately for librarians and non-librarians but found the difference in their ratings of off-line and on-line
searching not statistically significant even though the combined mean of the eight librarians rated on-line searching higher by .05
on a 7 point scale while the professional users rated on-line searching lower by .74. Librarians thought RECON more satisfactory and
worthwhile than off-line searching but less useful and productive. Professional users found RECON considerably less satisfactory, but
more worthwhile and useful and less productive than off-line searching. It seemed to the evaluators that librarians were annoyed by
the inability to build sophisticated Boolean expressions to isolate citations while non-librarians were annoyed by the restrictive
vocabulary that the system mandated, and everyone was annoyed by the slow response time. Wente, however, looked at the nearly 350
successful searches per week and concluded that “the test showed that working scientists and engineers both could and would use a
hands-on retrieval terminal, and further, that they could obtain useful results directly with no intermediary personnel and only a
minimum of instruction (Wente, 1971, 96). NASA was very pleased.
The project had cost Bunker-Ramo nearly $400,000. Just before the test began Dr. Ramo announced that he would step down as CEO and
Milton Mohr took over as President and CEO. Mohr then suggested that Mitchell find a different job and gave him some introductions
in TRW. At the end of the test NASA requested an extension at the same rental rate. Luther Harr, who was Executive VP, objected that
he had several new brokerage customers ready to pay higher fees to use the Univac 1050. Mohr then told NASA that they would have to
pay more than double “the rental prices on the consoles and communications gear involved, and NASA wouldn’t pay that much” (Mitchell,
NASA now had to take a half step back. Fortunately, Melvin Day and Roger Summit had continued to converse about the on-line
bibliographic retrieval system, now called DIALOG, that Lockheed continued to develop for in-house use on its new IBM 360/30. In
June of 1966 NASA awarded a $20,000 contract to Lockheed to “install and operate a remote terminal at NASA Ames Research utilizing
DIALOG to access the NASA file of 260,000 citations. The Ames Research Center is at Moffett Field in Mountain View, California,
within 10 miles of the Lockheed facilities and DIALOG project staff ”(Bourne & Bellardo, in-press). It took Summit’s group until
November to convert STI’s 1410 data files and build indices on the IBM 360 and then searching was instituted for 2 hours a day in
January 1967. Wente states that “this contract award was based primarily on three factors: the need to test IBM’s then new Data Cell
(2321) in on-line use; the availability of Lockheed’s independently developed computer programs at Palo Alto, and the real need at
the Ames center nearby for a local capability to retrieve from NASA’s information collection" (Wente, 1971, 96). NASA had successfully
hedged their bet on Bunker-Ramo by arranging a small-scale demonstration on state-of-the-art equipment and a proven search interface.
After a second RECON solicitation, this one based on NASA’s experience with both Bunker-Ramo and Lockheed, NASA awarded the second RECON contract to Lockheed in June of 1968.
In 1966 Rowena Swanson asked the question about the repeated failure of information systems in the context of calling for
better-planned systems that used the knowledge of technology experts as well librarians. Simon Ramo was the foremost expert in the
practice of systems engineering and Bunker-Ramo specialized in real-time information systems. But Bunker-Ramo was driven out of the
airline reservations business and walked away from the online information retrieval business, both for business reasons and business
politics. It is clear that the Direct Electronic Library approach could have succeeded, if it had been properly implemented, because
the systems analysis was sound. Because the Teleregister division was struggling to stay viable (it re-focused on the brokerage
industry and in 1968 signed a contract to build the NASDAQ system) it did not have the spare resources to implement a new large-scale
real-time project. A total of 8 employees worked on the RECON contract for different periods of time (not counting their support
staff, computer operators, etc.), and none were assigned to it full-time. The project team was compartmentalized on either coast and
not even Mitchell had good information about team performance and accomplishments. It is a testimony to the skill of the individuals
in the two groups that the system came together on time and worked as well as it did.
In September of 1965 Herbert F. Mitchell wrote in his proposal to Goddard: The typical new electronic information-handling system
cannot be designed best either by the customer alone, equipped though he may be to understand his own problems, or by an electronics
hardware company that builds its customer approach around the conventional marketing of computers. It is necessary to come to
understand simultaneously both the technology and the customer problem, and to envisage alternate ways of bringing a system of
people and apparatus together to do the job better. The problem must be quantitatively assessed in detail, and economic-technological
trade-offs for comparison of alternate approaches must be made (p. 6).
None of Summit, Borko, Kessler, Bourne, or Salton, took this approach to the online retrieval systems design problem. These men
were closer to academia and simply started small, used the resources immediately available, and sought to make their prototypes
useful to a local audience. These other pioneers enjoyed the time and freedom to tinker with their systems, experiment, revise, and
discuss them with colleagues at professional meetings. Herbert F Mitchell, like Simon Ramo, simply solved the current problem and
moved on to the next challenge.
What did the Direct Electronic Library leave behind? Meister and Sullivan published a single article and their contractor report. Those
are the only records published by participants other than Herbert F. Mitchell’s privately published autobiography. Van Wente published
one article from NASA’s perspective. Thus three articles have been the grist for scholars who have traced the development of online
information retrieval systems. Summit and others copied Meister and Sullivan’s approach to evaluating online information systems
simply because it was the first large-scale example. Sometimes the statistics published in the contractor’s report are mentioned
(Wente quotes them) but there is no mention of the conclusions not published in their paper: “In addition to its significant
performance in its primary role, RECON was successfully utilized to (1) update the very bibliographic collections upon which it is
based, (2) trace the evolution of work in a given field, (3) provide a chronological record of an author’s work, and (4) trace the
history of corporate involvement in a given field” (p. 13).
Lancaster and Fayen (1973) begin their review of Meister and Sullivan’s report with “Over a 7 week experimental period a staggering
6133 uses were recorded at six separate NASA centers” (p. 200). Other reviewers seem not to understand that no other online
bibliographic retrieval system accumulated that number of uses (or that number of hours) for at least two more years (for the first
80 hour DIALOG test at Ames Lancaster cites Summit’s report that the system was down 24% of the scheduled time and in all 96 searches
were completed, while in a subsequent 12 month trial at NASA headquarters only 13% of the 300 scheduled terminal hours were lost and
approximately 300 searches were conducted (pp. 192-196)). Although less than 40% of RECON searches were considered successful, that
still meant that in seven weeks nearly 2500 successful searches were conducted (we don’t remember Babe Ruth for setting the strikeout
It is easy to critique the system from the point of view of information retrieval, which is what the users did. The NASA personnel
wanted to find information (even though many were skeptical that what they were interested in what was in the library to begin with) and
they expected that the system would lead them to the information they wanted (NASA did not present this as an experiment but as a
new and improved tool). Salton and Borko were focused on the problem of automating the storage and retrieval of information, and
they crafted their systems to help the user extract knowledge. Mitchell and Summit focused much more narrowly on the problem of
retrieving documents via coordinate values. It was not their concern whether the indexing was accurate, what terms were in the
thesaurus (although Mitchell kept track of terms usage so that synonyms could be increased), or whether the locus of the user’s
input hinted at other pathways to the information he wanted to find.
Mitchell maintained that it was the purpose of the test to find out what users wanted in the system and how they wanted to use it,
so his focus was simply to build an (inexpensive) system that did what the contract called for, provide large-scale remote access
to the NASA citation collection with an easy-to-use interface. He would have welcomed the chance to refine the interface and the
search strategy and had programmed the system to keep the usage data necessary to guide revisions (the source of Meister and
Sullivan’s and of Wente’s statistics). Unfortunately, Bunker-Ramo made a business decision to abandon the program rather than
continue the development of their investment. Like SRI in 1963, Bunker-Ramo was in the contracting business, not the information
In the computer field a successful software test is one that finds (correctable) errors. In recent times Wente reviewed 30 years
of STI and presented a table of 8 major evaluations, ranging from 1967 to 1989 that helped the organization improve.
The management of any information program as large and complex as NASA’s scientific and technical information program justifies
serious efforts at evaluating its effectiveness. Table 6 lists eight of the principal evaluations conducted by or for NASA in this
area. Results of these studies were invariably useful, although specific changes made to the program in consequence are typically
difficult to identify. However, the 1967 study on the 1966 testing of an online retrieval prototype clearly occupied a major role
in issuance of the new NASA specification for an improved NASA/RECON system in 1968 (Wente, 1990).
The Direct Electronic Laboratory showed NASA what was possible and established a standard that NASA expected all future systems to
meet. In May of 1967 Mitchell became the Director of the Operating Computing Branch at Goddard Space Center, and just before he
retired four years latter he had the opportunity to use the DIALOG version of RECON. He easily understood the system and recognized
many features that he thought (mistakenly) that he had originated. He never realized how much duplicative design work in online
retrieval was going on outside his view during the two years he devoted to the Direct Electronic Library.
An Electronic Data Processing System for United Air Lines, Inc. (CP-227-1). (1965). Stamford, CN: The Bunker-Ramo Corporation, v1.
Borko, Harold. (1965). The Conceptual Foundationss of Information Systems(SP 2057), Santa Monica, CA: Systems Development Corp.
Borko, Harold & Burnaugh, H. P. (1966). Interactive Displays for Document Retrieval (SP 2557). Santa Monica, CA: Systems Development
Bourne, Charles P. (1999). 1999 Miles Conrad Memorial Lecture, NFAIS Annual Conference. Retrieved April 22, 2002 from
Bourne, Charles P. , & Bellardo, Trudi Hahn. (2002). A History of the Early Online Industry and Technology.
Brandhorst, W.T., & Eckert Philip F. (1966).Guide to the Processing, Storage, and Retrieval of Bibliographic Information at the NASA
Scientific and Technical Information Facility(N66-34085), College Park, MD: Documentation Incorporated.
Drew, D. L., Summit, R. K., Tanaka, R. I., & Whiteley R. B. (1966). An On-Line Technical Library Reference Retrieval System. American
Documentation, 17(1), 3-7.
Eklund, Jon (1994). The Reservisor Automated Airline Reservation System: Combining Communications and Computing. IEEE Annals of the
History of Computing, 16 (1), 62-69.
Englebart, Douglas C. (1986). Workstation History and The Augmented Knowledge Workshop. Proceedings of the ACM Conference on the
History of Personal Workstations. New York: ACM Press 87-100.
Lancaster, F.W. and E. G. Fayen. (1973). Information Retrieval On-Line, Los Angeles, CA: Melville Publishing Co.
Meister, David and Dennis J. Sullivan. (1967). Evaluation of User Reactions to a Prototype On-line Information Retrieval System,
NASA CR-918. Canoga Park, CA: Bunker-Ramo Corporation.
Mitchell, Herbert F. (1964). Unsolicited Proposal, Direct Electronic Library, A study of the use of Electronic Library Reference
Centers for the national Aeronautics and Space Administration, Bunker-Ramo Eastern Technical Center, Inc.
Mitchell, Herbert F. (1965a). Bunker-Ramo Direct Electronic Library, The Bunker-Ramo Corporation.
Mitchell, Herbert F. (1965b). Unsolicited Proposal for Technical Assistance to NASA Goddard Space Flight Center, (No. 5656-926).
Silver Spring, MD: Bunker-Ramo Eastern Technical Center, Inc.
Mitchell, Herbert F. (1966). NASA/RECON — Computer/Library at Your Desk. Canoga Park, CA: Bunker-Ramo Corporation.
Mitchell, Herbert F. (1990). My Biography, Cape Coral, FL.
Ramo, Simon (1961). The Scientific Extension of the Human Intellect, Computers and Automation,10(2), 9-12.
Schatz, Bruce R. (1997, January 17). Information Retrieval in Digital Libraries: Bringing Search to the Net. Science, 275, 327-334.
Simpson, G. S. Jr., & Flanagan, C. (1966). Information Centers and Services. In Carlos A. Cuadra (Ed.) Annual Review of Information
Science and Technology. New York: John Wiles & Sons. 305-335.
Sullivan, Dennis J., David Meister. (1967). Evaluation of User Reactions to a Prototype On-line Information Retrieval System.
Proceedings of 30th Annual American Documentation Institute, v4, Thompson Books, 90-94.
Swanson, Rowena. (1967). Information System Networks — Let’s Profit from What we Know. In George Schecter (Ed.), Information
Retrieval. Washington D.C.: Thompson Book Company.
Weik, Martin H. (1964). A Fourth Survey of Domestic Electronic Digital Computing Systems, REPORT N0. 1227 Aberdeen Proving Ground,
Md: Ordnance Ballistic Research Laboratories Retrieved September 15, 2001 at http://ed-thelen.org/comp-hist/BRL64.html.
Wente, Van A. (1971). NASA/RECON and User Interface Considerations. In Donald Walker (Ed.) Interactive Bibliographic Search: the
User/Computer Interface. Montvale, New Jersey: AFIPS Press.
Wente, Van A. (1990). Scientific and Technical Information Management. Government Information Quarterly, 7 (2), 149-167.
Return to Table of Contents.
Return to List of Appendices.