Dr. William Mitchell
Professor of Information Science
University of Arkansas at Little Rock


NASA RECON is acknowledged by most historians to be the first multi-site on-line bibliographic system for diverse literature. This paper examines the events that led NASA to contract with Bunker-Ramo Corporation for the development and test of a RECON prototype in the mid-60s. The design considerations of the prototype are described and the events explaining the test conducted in the fall of 1966 in four NASA research centers are chronicled. Even though the two-month trial proved the on-line bibliographic retrieval concept and NASA desired to continue its deployment, Bunker-Ramo withdrew and NASA had to re-advertise and restart RECON with another contractor. The reasons for these events are now revealed.

Charles Bourne and Trudi Bellardo Hahn deal with the development of on-line systems in their book (Bourne & Bellardo) much more comprehensively than should be attempted here. This exposition focuses narrowly on the events and circumstances of three turbulent years and deals with the disparate conceptions of the bibliographic retrieval systems that explain the diverse systems that were realized in this period.

Design and Development of On-line Bibliographic Retrieval Systems, 1964-1966

Addressing the Third Annual Colloquium on Information Retrieval in Philadelphia in May of 1966, Rowena W. Swanson, then deputy director of the Information Sciences Directorate within the Air Force Office of Scientific Research, discussed the engineering aspects of building information systems. "The words ‘information’ and ‘system’ each contribute meaning to the term ‘information system.’ Failure of information specialists to provide a definitional specification of systems concepts is all too prevalent and may indicate lack of understanding of them. This understanding is second nature to systems and industrial engineers and operations researchers”(Swanson, 1967, 3). Swanson enumerates examples of existing information systems for scientists, for industry and commerce, for managers, for the military, and for libraries, highlighting approximately 30 of approximately 700 that she estimated to be documented in the United States in 1966. She then turns to survey the efforts to create national networks, emphasizing published COSATI recommendations and the suggestions published in 1965 for a National Scientific and Technical Information Library System. At the end of this survey she asks what all the fuss is about. In a system, she notes, the whole is more than the sum of its parts, insofar as the parts, individually, are inert, needing each other to reach a nascent state. How closely related are the poorly shaped information systems of the past two decades to repetitive abortive phenomena? How many systems are planned, how many ad hoc? Is it the fault of the system or the environment in which it was born or not permitted to grow that a system cannot respond meaningfully to its interrogators (p. 26)?

Swanson concludes her extensive paper (over 200 entries in her bibliography) with a summary of the evident directions of applied and basic research in information systems that will be pursued by those she still considers to be pioneers of information science. In making the transition from retrospection to future prospects, Swanson addresses the conflux of technology and the professional library community. She does so by emphasizing the role that computers and systems technologists will play in future information systems.

H. F. Mitchell looked at the future of the computer in early 1965. He foresaw a remote on-line facility costing less than $100,000 (a $30,000 computer, a $1000 cathode-ray-tube console, and a $40,000 disk file) and time-sharing costing as little as telephone service by 1970. If the computer is, as Mitchell characterized it, an adjunct to the main objective of information processing and much of the equipment is on the shelf now, where are the information processing pioneers? Do information system designers in science and technology, and possibly managers and investigating groups, subscribe to the belief, expounded in the popular press, that automatic information handling depends on computers? The systems described in the foregoing sections disclose a new pattern of cooperation between groups having particular information problems and computer people leading to procedures that merge data base with equipment into a dynamic entity (pp. 26,27).

Not known to Swanson in May of 1966, Dr. Mitchell was but one of several non-librarian pioneers who were demonstrating on-line scientific bibliographic information retrieval systems that year. These pioneers, for the most part, knew little of each other. The currents of technology had by the middle 1960s simply made inevitable the evolution of on-line systems. What is of interest is the different perspective that each of these pioneers brought to the task of information systems building.

Swanson enumerated the existence in 1965 of numerous off-line information systems that employed mainframe computers in batch mode to search databases in response to queries. In terms of database size, the most significant of those focused on bibliographic retrieval were operated by NASA, which began in 1962 running bibliographic citation searches on an IBM 1401, and the National Medical Library, which made MEDLARS operational on a GE 235 in January 1964. Bruce R. Schatz, Librarian and Director of the Digital Library Research Program at the University of Illinois at Urbana-Champaign, in his review of the history of computer-based searching in 1997 for Science, characterized the early 1960s as the “grand vision” era and attributes it to the over-optimism accompanying the birth of artificial intelligence.

The first attempts at realization of the grand visions, during the 1960s, centered around text search of technical citations. The content was the text of a bibliographic citation of a journal article, which included the title, author, journal, and keywords of the referenced article. A search query was matching specified words to words in the fields of the citation. Prototypes of online systems for searching bibliographic databases were built in research laboratories in the early 1960s, and the most successful of these evolved through the large-scale experimental testbed phase into commercial services (pp 328,329).

Schatz credits the Council on Library Resources studies, culminating in Licklider’s 1965 report Libraries of the Future, for focusing the vision, but as Swanson had articulated, there were many spokespeople in the 1960s with a vision of a computer-mediated information rich world. One such visionary was Simon Ramo, who published a 1960 talk he gave entitled “The Scientific Extension of the Human Intellect” (Ramo, 1961). Dr. Ramo was a distinguished engineer and co-founder of Ramo-Wooldridge, which had been responsible for systems analysis and management of the Air Force’s ICBM program in the previous decade. He portrayed a future where communication and computer technology would support knowledge communities, allowing easy on-line access to experts and cataloged expertise where all participants critiqued and augmented the community knowledge store. Dr. Ramo believed that such a future was within reach of the technology that his firm was developing for the military. Swanson also knew the nearness of such a vision of man-machine collaboration as she had funded Douglas C. Englebart at the Stanford Research Institute (SRI) since 1962 in his Augmenting Human Intellect project. Englebart (1986, p. 75) remembers that the first annual report under that contract “with the considerable help of Rowena Swanson, …was condensed into a chapter of a book published in 1963 [Vistas in Information Handling, Howerton and Weeks (Editors), Spartan Books, Washington, D. C., 1963, pp. 1-29]”. Herbert F. Mitchell worked for Simon Ramo.

In 1965 there were no on-line information systems that merited the designation “system”. Roger Summit had built a prototype “conversational” inquiry machine based upon library catalog cards, called Converse I that he described in July 1965. He wrote the data from 8000 catalog cards sequentially on magnetic tape, created inverted files on each of the significant title words and citation descriptors, and then punched the records of the inverted entries to cards (38,285 of them) and stored the punched descriptor cards in cabinets adjacent to a card reader and a teleprinter. A search was conducted by selecting descriptor cards from the cabinets and constructing a deck that would isolate articles satisfying all descriptors (with the option of interspersing NOT cards to change the sense of the descriptor).

Place the first inquiry card, with letters right-side up and facing the user, in the long card slot on the top of the machine. Enter the inquiry card by depressing the metal bar on the back of the card slot. Repeat the procedure for each new card input. When all the inquiry cards have been entered, insert an END card” (p 4).

After the END card was read and if three or fewer satisfied the query, the machine would print out the isolated catalog cards. If 4-15 catalog cards satisfied the query only the card reference numbers would be printed. The user could then force the printing of those catalog cards or add additional descriptor cards to narrow the search. If more than 15 cards satisfied the query only the total number was printed and the user was expected to continue to reduce the number by specifying more descriptors (hence the “conversation” would continue).

The Lockheed Electronic Sciences Laboratory in mid-1965 had plans for Converse II that would eliminate the card reader and allow instead for the user to input terms on a teletypewriter. Supposing that the catalog cards for all 100,000 of Lockheed’s in-house report collection were stored on tape, the user’s term would be matched against the descriptors and the response would be the creation of a subfile of the article identifiers whose size would be output on the printer. The user could create any number of such subfiles and would also have the capability of manipulating the subfiles by Boolean operations that would merge and intersect the identifier lists, each time reporting the size of the resulting file. All intermediate subfiles would be preserved to permit backtracking at any time during this interactive search. Converse III was foreseen to use a CRT for output.

In mid-1965 Lockheed knew only of experimental on-line bibliographic retrieval systems at MIT and at Harvard. In Summit’s view Salton’s SMART system was a very interesting experiment, but it existed only in the lab and had no users. The TIP system at MIT was already using teletypewriters (IBM 1050 communications terminals) for input and was accessible from over 100 such terminals on the MIT campus starting in January 1965. The TIP system searched a collection of over 30,000 articles found in several years of 25 Physics journals. Lockheed felt that the TIP system design was biased toward journal article retrieval and would not work as well with a multi-disciplinary report collection (one of TIP’s unique features was retrieval of all articles in the collection that cited the same sources as a selected article).

Paired with Swanson in the first workshop session, H. P. Burnaugh reviewed the BOLD system that he and Harold Borko developed in 1965 as one of the projects of the Language Processing and Retrieval Staff at System Development Corporation. Although conceived as an experiment and developed on a Q32 military computer with capabilities not then commercially available, BOLD permitted a dozen workstations connected to the mainframe at SDC via the SDC Time-Sharing System to search a collection of 1745 abstracts from the Defense Documentation Center using any combination of 6883 retrieval terms (Borko & Burnaugh, 1966). The BOLD user began a search by typing BEGIN/ and then entering any search term he choose followed by a question mark. The system retrieved all similar terms in the dictionary and listed them along with the number of articles each referenced. The user would repeat this process to learn the vocabulary of the collection and would then compose a query specification by typing phrases that interspersed dictionary terms with AND, OR, and NOT. When the specification is complete, the user commanded SEARCH/ and the system applied the query specifications to the data base and returned the number of citations satisfying all the specifications as well as a table that showed each article reference ID and which of the dictionary terms it was associated with. The user could delete rows from the table with a light pen and obtain any of the information on any of the articles, including its abstract, by switching to browse mode. By commanding TYPE DISPLAY/ the contents of the CRT would be sent to the printer.

The BOLD system made use of $100,000 CRTs and light pens as well as a computer comparable to an IBM 360/65. SRI and Englebart had also used the Q32 two years earlier to demonstrate on-line information retrieval, as reported by Charles Bourne (1999):

"The first demonstration of an online bibliographic and full-text search system, and remote online searching, was a system developed and demonstrated with Air Force and ARPA support at Stanford Research Institute (SRI) in 1963. This was done as an adjunct project to the Augmented Human Intellect Program that was then underway at SRI under the direction of Doug Englebart. As Project Leader for that online effort, and working with a very talented programmer, Len Chaitin, I tried to incorporate and make use of everything that we had learned from the almost 20 operational batch search systems that had been demonstrated by that time. We felt that this online approach was a natural and logical extension of what had already been done with the batch search systems. Our system made use of programs and a database of full-text and bibliographic records that were prepared at SRI, and then installed and searched on the Q-32 computer at SDC-Santa Monica from a small computer workstation (CDC-160A) 400 miles away at SRI."

In 1963 the Defense Department wasn’t interested in following up on the demonstration.

Borko, writing in May 1965, had not yet seen anything published about Converse I. “Prototypes of parts of such an automated information center already exit at MIT, as part of Project MAC (Multi-Access Computer); at Harvard in SMART (Salton’s Magical Automatic Retriever of Texts); and at System Development Corporation in the BOLD (Bibliographic On-Line Display) System” (p. 30).

Dr. Herbert F. Mitchell and the Direct Electronic Library

Dr. Mitchell received a Ph.D. in Applied Mathematics from Harvard University in 1948 where he and Grace Hopper worked under Howard Aiken to build the Mark II. He joined the Eckert-Mauchly Computer Corporation in December of 1949, just before it was assumed by Remington Rand, and headed the programming department for the first UNIVAC I. In 1953 the Univac Division was reorganized and John E. Parker was made VP of computer sales with assistant Luther Harr. Dr. Mitchell was assigned to Parker as the senior technical person for computer sales for business applications, responsible for sales demonstrations and more than 200 programmers who programmed the demonstrations and did service bureau work. Dr. Mitchell held various sales support positions in the company after it merged with Sperry, ending in 1960 as assistant director of product planning. In rapid succession he became Systems Sales Manger for the Eastern US for Honeywell, then Honeywell’s Western Regional Sales Manger, and in 1961 was hired by Art Collins, founder of Collins Radio Company, to participate in the design of a new switching computer, the C8401. A year later Parker, who in 1962 became the Chairman of the Board of the Teleregister Corporation in Stamford, Connecticut, invited Mitchell become the Vice President for Advance Research starting in September 1962. Parker had already recruited Luther Harr as VP for Sales. In 1963 Parker became President and CEO, so the old team was together again.

Telegister Corporation had been a subsidiary of Western Union until 1948 when it was spun off as a public corporation, but in 1962 it still had less than 200 employees. It manufactured tote boards for the brokerage industry and got into the airline reservations business with American Airlines in 1945, building a pilot seat inventory system that became operational in 1946 in Boston. The prototype was followed in 1952 by a more comprehensive seat management system installed in the New York office. Teleregister built the special purpose computer that ran the system, accessing a large drum, and used second generation computer technology so successfully that eventually Teleregister handled seat reservations for TWA, United, Braniff, National, Northeast, Pan American, Western, and several foreign carriers. Jon Eklund, Smithsonian Institution curator in the Division of computers, Information and Society, concluded his 1994 review of this accomplishment with the words: "Reservisor was the first commercial system to combine electronic processing and electronic communications. Because it ran so long and so reliably (the equipment operated 22 hours a day, seven days a week, with a two-hour daily maintenance period), it provided a testbed and standard for large-scale heavy-duty information-technology applications. Resevisor demonstrated not only that information could be reliably processed by a central computer, but also that it could be processed at the time the transaction occurred or with negligible delay, and that the system could process information with input stations scattered over a wide geographical area" (pp. 68,69).

IBM became a reservations competitor in 1958 when it contracted to develop the SABRE system for American Airlines (that became operational in 1964), and in 1962 Univac was selected over Teleregister to handle reservations for Eastern Airlines. Teleregister, it seemed, was viewed as too small to be depended upon for mission-critical applications. While struggling to stay in the airlines reservation business by upgrading its customers from seat inventory systems to passenger reservations management, Teleregister capitalized on its real-time communications expertise and its Telefile computer to implement three completely integrated large-scale, on-line banking systems for the Howard Savings Institution, Union Dime Savings Bank and the Society for Savings, all installed in 1962. Teleregister also extended its dominance in the stock brokerage industry with the Telequote II system that put market data on a broker’s desk. More than 1000 subscribers across the country were online in May of 1964. That year fourteen Telefile computers were operational at six different sites (Weik, 1964).

Mitchell managed the research staff at Stamford and pursued ideas and products that would lead to new customers. In mid-1963 he worked on a study funded by the National Security Agency that resulted in the specification of a new computer, the Teleregister Control Processor, designed to effect information interchange among all the different computers at NSA’s computer installation at Fort Meade, Maryland. NSA decided instead to move to IBM computers. In late 1964 he managed the development of an Automated Program/Budget Control System for Air Force Headquarters. In early 1964 he worked on a proposal to update the United Air Lines Instamatic reservations systems that Teleregister had installed in May 1961. In his autobiography (Mitchell, 1990) he recalled that:

"United wanted the computer system not only to handle the complete passenger reservation system (similar to what is now used by all airlines), but also to track all the other operations within the air line — freight, flight schedules, pay, maintenance, profit and loss, and even load and balance and fuel requirements for every flight. All of this for 1,500 flights per day, 250 airports, and 2,500 pilots. I had charge of the preparation of the technical portion of the proposal. In order to learn the details, United had me visit their operation centers at New York City, Chicago, and several smaller centers. I had to learn how crews were scheduled (including all union contract terms), how pay was calculated (different rates depending on whether the airplane was at the dock, on the taxi strip, in the air over land, or in the air over water). I had to learn how priorities were handled for the loading of the plane — passengers, freight, mail, special cargo, baggage, etc. Finally, I had to figure out how to get a computer to keep track of all these details in real time, and do so in spite of individual component failures. It was a tremendously interesting assignment, and I was proud of the result" (p. 146).

United’s management liked the proposal so much that it used it as a basis for an RFP to the industry in late 1964, and then in December 1965 choose Univac instead of Teleregister to implement the new reservations system.

"John Parker sat on the Board of Directors of Martin-Marietta Corporation, and in early 1964 he learned that CEO George Bunker and Bunker’s long-time friend Dr. Simon Ramo, who was then vice-chairman of the board of another Fortune-500 aerospace company, TRW, had arranged to purchase a division of each company (financed by Martin-Marietta) and create the Bunker-Ramo Corporation “with the objective of providing a unique corporation to fill what was seen as a national need in the application of electronics to information handling” (Mitchell, 1965b, 5). Parker immediately suggested that Teleregister be added and the merger of Teleregister and Bunker-Ramo was affected on July 8, 1964, creating a company with three divisions under CEO Simon Ramo and Chairman Parker: the Computer Division of Thompson Ramo Wooldridge, Inc., the Electronics Systems and Products Division of the Martin Marietta Corporation, and the Teleregister Corporation, with 3,100 employees and a combined annual sales of approximately $60 million. “It was believed at the time of the formation of Bunker-Ramo that the electronics industry in this country particularly interested in computers, displays, and other aspects of electronic information-handling largely emphasized hardware; industry failed to meet the requirement of equal emphasis on systems engineering” (Mitchell, 1965b, 5-6). The new company sought to engage in certain specialized and key hardware areas in electronic information handling and in the systems engineering of information handling.

Mitchell’s family resided in Los Angles and he had been working away from home for two years, so he immediately asked to be transferred to the Canoga Park division. “Si Ramo spent a good deal of his time with the new company, and I became well acquainted with him. My title was changed to staff vice president (whatever that means), and I was given the mission of developing the commercial side of the division’s business, which had become almost exclusively Federal-Government oriented” (Mitchell, 1990, 148). Dr. Ramo was acquainted with Melvin Day, then director of the Technical Information Division at NASA, and was aware of Day’s interest in developing an on-line bibliographic retrieval system at NASA (more below). Ramo and Mitchell talked about the similarity of batch bibliographic retrieval, the existing scientific currency awareness services, and airline reservation retrieval and concluded that Mitchell should draw up a proposal to NASA. Mitchell completed this task in the early fall of 1964 and the 14-page unsolicited proposal for a Direct Electronic Library (Mitchell, 1964) was immediately delivered to Melvin Day.
The emphasis in the report was on the cost of setting up a nationwide network to connect 15 NASA installations employing 34,236 personnel, one-third of which might make use of the library, and thus requiring the distribution of 102 CRT consoles among the installations. In a single center configuration these CRT consoles would communicate with a single computer and its disk file controller, which limited to 100 the number of concurrent searches that could be accomplished across the system. By increasing the number of centers in a network the proposal showed both that the number of parallel searches could be increased as well as the volume of citations stored on-line, but anticipated a corresponding increase in demand for access. A single center system was projected to cost $54,500 per month and provide for 54,500 searches per month to NASA at the targeted $1 per search, with a peak capacity of 350,000 searches. A three-center system was estimated to cost $108,500 per month, hence providing 108,500 searches at $1 each, but having a peak capacity of 1,050,000 searches per month. If 12 centers were configured, the cost grew to $349,000 per month with a peak capacity of 4,200,000 searches (over 5 searches per workday for every NASA employee).
The search strategy was described in a single page, first defining a direct reference, the list of accession numbers returned by specifying a single coordinate value (author, sponsor, etc, perhaps within a date range), and then indirect references, a list of accession numbers generated by user-chosen key words.

The computer at the center will attempt to match each such key word against a master list containing all indexed key words for the chosen category plus all anticipated synonyms of these. As each successive key word is entered, the computer responds with a rejection indication if it fails to identify the key word, or the number of articles which are associated with that key word (or its synonym) and all earlier accepted key words. Further restrictions upon selection may be imposed by specifying inclusive dates of publication, role status of the key word, type of article, size of article, sophistication of treatment, etc., as is deemed desirable. Each specification adds to cost of indexing, of storage, and of selection. When the number of isolated articles has reached a sufficiently small value, the user requests display of titles, and then proceeds as described for a direct reference (display of the full citation and instructions for ordering the full article] (p. 5).

To Ramo’s and Mitchell’s knowledge in 1964, no on-line searching was being done, but they saw no technical impediment to constructing a large-scale system for NASA that would permit coordinate searching of bibliographic citations by over 1000 simultaneous users nationwide with reasonable response times.

Bibliographic Information at the NASA Scientific and Technical Information Facility

NASA initiated a program to disseminate bibliographic information about the aerospace literature in 1962 by contracting with Documentation Incorporated to abstract, index and publish an abstract journal entitled Technical Publications Announcements that was renamed Scientific and Technical Aerospace Reports (STAR) in 1963. STAR focused on report literature generated by or of interest to NASA and was complemented by a similar NASA-supported service provided by International Aerospace Abstracts (IAA) that covered the journals, books and other “open literature” of aerospace. “The two journals together attempt to provide comprehensive access to the world’s current literature dealing with aerospace science and technology. By special arrangement, STAR and IAA are issued in coordination with each other and on alternate weeks. Both use identical subject categories to group their announcements, and both contain basically the same indices” (Brandhorst & Eckert, 1966, 3). From January 1962 through December 1965 the bibliographic citations converted to magnetic tape by the Scientific and Technical Information Facility (STI) grew to well over 200,000. Batch searches using inverted files were performed on this collection using an IBM 1401 until the Linear File Search System became fully operational on an IBM 1410 in December 1964. NASA offered the tapes and several organizations accepted, producing their own programs to manipulate the tape data according to the needs of their own particular operating environment. Most of these operations involve severely re-formatting the NASA file with considerable loss of the non-subject data the Facility finds necessary. They make no attempt to be the equivalent of the Facility’s present 1410 system, being contributions to the art in their own right. Three of these efforts, each involving IBM-7090 series equipment, at the University of Pittsburgh, Republic Aviation, and North American Aviation, are described in References 12, 17, 20-22. Additional activity, involving a CDC computer, is currently going on at the University of Indiana [the references are dated from December 1964 to December 1965] (Brandhorst & Eckert, 1966, 14).

None of the previously mentioned pioneer on-line systems apparently were involved with NASA’s efforts.

A batch searching service was offered by STI and Brandhorst reported in 1966 that it “has prepared, edited, and delivered over 1000 searches in 1965, all of which, if not exclusively machine searches, received their major contributions from a machine search” (p. 34).
Imagine Day’s excitement at Bunker-Ramo’s suggestion that over 50,000 searches per month could be conducted from any NASA facility in the country at a cost of $1 each!
Van A. Wente directed systems development at STI and plotted the strategy for introducing on-line retrieval (called the Retrieval Dialog Study (Simpson & Flanagan, 1966, 318), this could have been the motivation for Roger Summit to rename Converse II). He notes NASA had already placed in many field center libraries duplicate copies of the documents indexed at STI. “The documents, in nearly every case, could be viewed immediately either as journal articles, books, proceedings, full-size reports, or microfiche back to 1962” (Wente, 1971, 97). These duplicate libraries were under-utilized because of the effort involved in manual searching the paper indices, so the missing piece was on-line citation searching, which NASA had already decided to call RECON, short for REmote CONsole (Bourne & Bellardo, in-press). Dissemination of access to the citation collection at STI, precisely as Mitchell’s proposal described, would permit in most cases not simply the identification of a document that could be delivered in several days, but the alerting of a local librarian who could supply the document within the hour! “Indeed, any failure to have full document text available would clearly break the full chain of feedback and iteration which on-line systems usually employ” (Wente, 1971, 98).

Bourne reports that during the fall of 1964 Roger Summit had occasion to discuss his ideas about Converse with Melvin Day and he followed up with an unsolicited proposal to NASA to extend Converse to the NASA collection (Bourne & Bellardo, in-press). Summit, like Borko at SDC, Kessler at MIT, and many others in the early 1960s were working in experimental time-sharing environments. In contrast, Mitchell was proposing a real-time dedicated application derived from years of experience in seat reservations, stock market quotations, and on-line banking, and employing existing commercial technology. NASA knew that there were still other organizations with interest and technology appropriate to mounting a bibliographic retrieval system of this scale so Wente developed specifications that in April 1965 resulted in NASA’s issuance of a request for proposals to develop a prototype system employing the full NASA collection, then about 200,000 documents, in a realistic environment of research libraries with direct use of the system by working NASA scientists and engineers. By a formal competitive process, NASA selected the Bunker-Ramo Corporation to conduct the test principally at three NASA locations using remote terminals and programs owned by that company and operated through a UNIVAC 1050 computer located in New York City (Wente, 1971, 96).

Day apprised Ramo of the decision to issue an RFP so Mitchell began work on the winning proposal in February of 1965. He continued to call the system the Direct Electronic Library because NASA had not shared its chosen name. He published his 18-page prototype proposal dated April 12, and distributed it to various company officials ten days later, including Si Ramo on April 23, 1965 (Mitchell, 1965a). During those ten days he developed the algorithms that demonstrated that the features described could be performed. In contrast to the previous emphasis on systems level communication and cost, the prototype devotes all but three pages to describing the user’s search interface as was requested by NASA’s RFP.

Mitchell’s charge was to find commercial applications for the expertise of the Canoga Park Division. He had Ramo’s support that there was a commercial future to on-line information retrieval and his idea was to develop a system that would permit users to select any of a variety of information databases, combining all of the existing medical, legal, and scientific information services into one system. "NASA’s RFP seemed to me to be an ideal way of getting started in the broad field I had been proposing, and I persuaded our management to bid on the contract. I also persuaded management to do the programming without charge so we could have proprietary rights to it when the contract was completed. Otherwise the government would own those rights. Of course, we won the contract, as our price was so much lower than the other bidders, and we jumped in to get the job done" (Mitchell, 1990, 149).

In the prototype proposal the 40 lb Bunker-Ramo 213 CRT terminal is described as the user console with its row of 16 function keys re-labeled to invoke search actions (however, the picture shows a terminal with function keys labeled for an airline reservation application). The user begins a search by selecting the collection of citations to be searched.

For the average user it is anticipated that several or many collections will be available through his console, some perhaps requiring proper identification on his part before access will be granted by the System. The user initiates his reference by depressing the “Collection” function key, followed by the keys describing the desired collection, and terminated by depressing the “Transmit” key” (Mitchell, 1965a, 7).

If the entry was not recognized, the response is a numbered display of all of the available collections, whence the user chooses a displayed collection by pressing the “Collection” function key followed by the number of the list entry, followed by “Transmit,” or possibly continues to the next page of the list by simply pressing the “Collection” function key.

Once the collection is established the user can isolate relevant citations by using the “Report number” function key, the “Accession number” function key, the “Author” function key, the “Sponsor” function key, the “Keyword” function key or the “Date Range” function key. In all cases, should the entry following the function key not be recognized by the System, a numbered list of acceptable entries is presented, and if the entry is recognized, it is added to the specifiers already accepted and a numbered list is displayed showing each accepted specifier and how many citations in the collection remain when this last specifier is imposed in addition to the previous ones. Keyword entries are sometimes subject headings and when this occurs the response is a numbered list of the terms subsumed by this heading. If the entered keyword is not recognized a list of index terms with the same stem prefix will be returned.

Provision was also made to “or” into the working set citations isolated by additional author, sponsor, or keyword specifiers by means of inserting a “/” between the function key and the specifier (whence, if recognized by the system, it will be listed as an ALTERNATIVE keyword and the size of the working set would usually increase with its addition). Provision was also made to erase any specifier in the list of specifiers, nullifying its effect. “The erase feature is particularly useful for examining several sets of titles based on a series of specification sets, of which one term only is varied. This process may be repeated as often as desired” (p. 13). At any time the user may press the “Title” function key and see a numbered listing of the full citations in the current working set (each one possibly requiring multiple screens). The “Page” function key is used to view the next screen, or, followed by a number, to jump forward or back to the designated citation entry. Should the user find articles he wants to see, he presses the “Deliver To” function key and follows with an entry identifying the delivery information. If the “Deliver To” function key is pressed when a subsequent citation is being displayed, it can be followed immediately with the “Transmit” key and the same delivery information is assumed. Pressing either the “Terminate” or the “Collection” function keys will reset the terminal for a new search.

In addition to citing the 100 concurrent user limitation in several places (the single center configuration) the prototype proposal cites a maximum response delay of three seconds when the line load is less than 70% of capacity, and promises that the planned installations will average only 50% line loadings. It also enumerates ten statistics that the System will compile, including hardware usage and failures, frequency distributions of keywords employed by user, referenced and un-referenced index terms, number of specifiers per search, citations referenced and un-referenced, and new keywords and accessions entered per time interval.

This document was given to the Eastern Technical Center (the Teleregister Division) to refine as Bunker-Ramo’s response to NASA’s RFP. No doubt the proposal and contracts people in Stamford included some of the same verbiage they had included in the United Proposal in April of 1965, such as "The Bunker-Ramo Corporation considers the development, design, implementation and operation of on-line, real-time systems its primary business and had specific success in managing, developing, manufacturing and operating extensive large-scale, on-line systems operating in real-time. The Bunker-Ramo Corporation has had more experience than any other corporation in the installation and maintenance of large-scale on-line systems in business and industrial applications (An Electronic Data Processing System, 1965, 2,4).

References to the past accomplishments of both TRW and Teleregister together with a proposed price of $235,000 for the large-scale demonstration offset the fact that the Direct Electronic Library existed only in Dr. Herbert F. Mitchell’s imagination. NASA was in the position of selecting from among proposals from organizations with working on-line retrieval prototypes but no experience with large-scale, real-time applications, and a proposal from a company that had an excellent reputation for delivering exactly this kind of system but which had not built a prototype. NASA awarded contract NASw 1369 to Bunker-Ramo in June of 1965.

The Bunker-Ramo Version of RECON

Milton Mohr, the head of the Computer Division in Canoga Park, was not pleased with the contract. The work was to be done by Teleregister Division people (Dr. Mitchell being one), on Teleregister equipment, and it was under-priced with the difference to come out of his budget. He refused to spend any of his R&D budget on the project and left Mitchell to scramble to piece together the necessary resources. Dr. Mitchell teamed with Dr. Dale Scarbrough, an electrical engineer who came with Simon Ramo from Hughes Aircraft when Ramo-Wooldridge was formed in 1953 and who now held the title of Senior Member of Technical Staff of Bunker-Ramo, to get the contract done. Mitchell recalls that Dr. Ramo was not so interested in the NASA contract as to protect Mitchell’s time.

"In 1964, Dr. Ramo had been given the Air Force’s award as the man who had made the greatest contribution over the first ten years of the Air Force’s guided missile program. This gave him considerable prestige in space circles, and he apparently decided to make use of it in September 1965. He had me present a proposal to the Goddard Space Flight Center for Bunker-Ramo to provide computer expertise to the space center as it installed its $25,000,000 order from IBM of their latest and fastest model 360 computers…I and my colleague in the information retrieval contract (Dr. Dale Scarbrough) were proposed as the super-experts. I learned later that Dr. Ramo had told Dr. Clark, Goddard’s director, that I was one of the most knowledgeable people in the business! Dr. Clark bought the idea, but before it could be implemented he was transferred to NASA Headquarters in Washington, and the temporary director was persuaded by the operating management that such an arrangement would be too difficult for them to handle" (Mitchell, 1990, 149).

In the Appendix of the proposal (Mitchell, 1965b) Mitchell’s vita identifies him as staff vice president “responsible for the development of new applications in the areas of real-time systems and information retrieval. Developed a system for remotely searching a large technical literature index using subject specification terms, known as the Direct Electronic Library” (p. A1). Scarbrough’s vita says “currently active in systems design of nation-wide remote interrogation of complex technical information data banks” (p. A4). The proposal was time critical and promised “each of these individuals would spend over the next six months a considerable fraction of his total time on the Goddard problem. Drs. Mitchell and Scarbrough would call on other specialists largely at Canoga Park to furnish specific analyses, judgments,...” (p. 9).

Mitchell devoted as much effort as he could spare to the NASA contract over the next year but it wasn’t his only responsibility. He contacted Mel Day and made several visits to the data processing center at Suitland, MD to arrange for the magnetic tapes containing the 200,000 citations and documentation as to how to decipher them. This was a non-trivial task because NASA had never attempted to deliver the entire database to a non-classified organization. STI had to write a program to identify all of the classified documents on their tapes and overwrite those entries with spaces. They then they had to feed their tape library through the program. The IBM 1410 was a character machine and the citation records were each variable length in three segments delimited by reserved characters. Mitchell also arranged to acquire part of the effort of three programmers from the group he supervised the previous year on the Air Force Budget project to begin the design of the Univac 1050 programs. In Canoga Park Mohr permitted the project to use the GE 425 computer that did the division’s business data processing on the third shift. A programmer was found and set to working on a program that would read STI’s 1410 tapes on the 24-bit word GE machine and output inverted files for the Univac 1050, a 36-bit word machine. This task was further complicated by the fact that the GE assembler was not well documented and the tapes were not formatted reliably (NASA contributed extra funds to compensate for the over two months of delay the errors caused).

During the last six months of 1965 Mitchell and Scarbrough flowcharted the system and discussed the search interface occasionally with colleagues. They viewed the job as the straightforward identification of a citation satisfying criteria and they knew how to efficiently manipulate inverted files. The search strategy was simply a logical interaction to input criteria values, so they needed little help in creating an efficient interface (in style like a reservations interface). Their goal was to permit the isolation parameters to be input as quickly and error-free as possible.

Other factors influenced the design of the search interface so that it evolved from that described in the proposal. The console was upgraded to a Bunker-Ramo 203 Display Station that had a larger screen and a detachable keyboard but fewer keys. As a consequence of learning more about NASA’s work environment, the search procedure now began with the user identifying himself by depressing the “User” function key and following it with one or more lines of identification terminated by the “Transmit” key. The System (now called RECON) responded with a numbered list of the series that the user was authorized to search (such as STAR N-10,000) and the user selected one or more by depressing the “List” function key, following it by list number or numbers separated by commas, and ending with “Transmit.” Input of lists was a new feature, as was avoidance of error in specifying the citation collection via selection from a user-specific list.

RECON then displayed the status of the “login” by echoing the user’s identification and chosen accession series and was prepared to accept specification terms: Author, Corporate Sponsor, Report or contract #, or Subject keyword. As before, an entry that did not match a term in the index list provoked a numbered “near” list of the valid terms, but now a valid entry also provoked a near list unless the function key had been pressed twice before entering the term. The rationale was to show more possible terms of interest that could be included in the specification or to allow the user to substitute a near term for the original one. When the near list is displayed, the user had to select one or more position numbers (the list may be scrolled up or down with the + and – keys) by again pressing the specifier function key followed by the position numbers selected (ending every input with “Transmit”).

In the event that subject keywords were being selected and one or more happen to be cross-referenced, RECON would interrupt itself and display a new list showing each of the keywords just chosen, each followed by the number of citations it referenced, and then, appended to this listing, a listing of all the cross-referenced terms, numbering from 51. The user made another selection from this augmented list by following the “Subject” key with the number(s) of the desired term(s) (original or cross-referenced), and, if no more cross-referenced terms were selected, RECON echoed the status of the selection criteria by incrementally adding the selected terms to the Search Status display and permitting the user to confirm or exclude its application.

In the Search Status listing the terms selected were displayed in a numbered list, each followed by the number of citations so far isolated in the working set. The user pressed "+" followed by the specifier key to accept the last displayed term, and if so, the next term was displayed with the number of citations that would result when its references were “or’d” into the working set. If the user rejected a term by pressing "–" followed by the specifier function key, that term was dropped and was replaced by the following term in the list with the process continuing until the list of terms chosen by the specifier function key was exhausted (END was displayed by RECON on the last line of the Search Status listing). The user could then “and” additional specifiers by pressing another specifier key and entering its value, could enter an “and not” specifier by first pressing "–" and then the specifier function key followed by a value, or could “or” in alternate references by pressing "+" and then the specifier function key followed by a value. The number shown after the last numbered entry in the Search Status screen was the size of the current working set and the user was always given the opportunity to see how the size would change before confirming the application of a new specifier. If the user reviewed the search status listing and wished to delete any specifier, he entered "-", followed by the list number of the specifier and “Transmit” and the search status was updated to show that the term had been removed and the new size of the working set as determined by the remaining specifiers.

When the user wanted to see the citations isolated in the working set he pressed the “Title” function key and “Transmit.” The citations in the working set were be displayed beginning with the most recently accessed and continuing through to the oldest, and the user could jump down the list by pressing “Title” repeatedly, or order the current citation by pressing the “Order” function key. When the user was finished with the working set, he pressed the period key (.) and “Transmit.”

Two other functions were provided: “Message” to dump the search status to the default librarian’s printer (the only provision made for printing the CRT display), and “Instruction” to access a tutorial, or, if followed by a second function key, to receive help about the use of that key (without affecting the state of the search). There was also a date key that accepted a range parameter and limited the working set to citations that satisfied that range. Date restriction, specifiers entered in the suppression mode (-), “ANDed” specifiers, and “ORed” specifiers were all applied to the whole working set.

Mitchell completed the conversion of the initial 200,000 citations (filling over 50 magnetic tape reels) in June of 1966 and personally delivered them and a users manual dated May 2, 1966 (Mitchell, 1966) to the Teleregister teleprocessing center in New York City. The manual described from the user’s perspective what the flowcharts had specified that the program should do. Mitchell continued to devote himself to other tasks. For the next five months STI sent update reels every two weeks to Canoga Park to be processed and then shipped to New York City so that the scheduled demonstration would provide access to the most current information.

On the east coast the programming team had been working from the flowcharts and had coded about 60 modules that had to be debugged now that the data was available. However, the Univac 1050 was providing services to Teleregister Division customers during the day so the programmers could only access the machine at night and on weekends. Site preparations were begun so that 23 consoles could be installed in six NASA centers, eight at Langley and six at NASA headquarters. NASA provided idle telephone circuits that were serviced by Western Union and a seven-week trial began in October 1966. Searching was supported all day long in the foreground partition of the Univac 1050 resulting in 800 or more searches a week, but bad telephone connections precluded the more distant sites from communicating with New York City. Bunker-Ramo had arranged for David Meister and Dennis Sullivan from their human-factors group in Canoga Park to conduct a formal evaluation of the trial and their report was both delivered to NASA (Meister, 1967) and also published (Sullivan, 1967). Both had assisted Mitchell with refining the design of the RECON user interface and they had made the rounds to train and interview librarians and NASA engineers at all of the sites. Because 40% of the searches occurred at Langley, most of the systems evaluation was based on data that they collected at that site.

However, the tested system was not quite the same system that Mitchell had described. The users manual for the test (Meister, 1967, Appendix D) shows that several modifications that were made after Mitchell delivered it, some merely editorial and some substantive (these documents were hand typed). On page 7 (and pages 9 and 22) it is noted that the “AND NOT” (suppression via "–", “Subject”, term entry) was not available. On page 11 (and page 23) deletion of previously applied specifiers is restricted to the last applied specifier. On page 17 the bypass option (selecting after pressing the function key twice) is not available when selecting from a near list. On page 20 the feature of invoking the current search status is added (previously the Search Status was a System response to a change in the working set specification). On page 26 the “Message” action is missing. On page 27 the specifications that error messages would include reference to the tutorial are deleted as is all mention of a busy message that was to appear, prohibiting the initiation of a search when the system was overloaded. Apparently the programming team had encountered problems coding all of the flowcharted algorithms.

As the testing proceeded, users discovered that RECON did not behave as presented in their manual (actually, the manual was ambiguous). The evaluators note that “when the OR function was employed, the term which had been ORed was locked to the immediately preceding term in that logical statement and could not be dissociated from it” (Meister, 1967, 14). In describing the step of entering an alternative specifier (prefixing a specifier function key with "+") on page 11 the manual says “It may be desirable to enter a specification term on an either/or basis with the previous specification term [emphasis added]. As we have seen, it is convenient to do this by multiple selection from an alphabetical or cross-reference list, but this is not always possible”. When demonstrating this function on page 21 the manual notes that the illustration shows an increase in the size of the working set. “This is because the new term was “ORed” to the preceding terms [emphasis added], which means that the search for documents referencing MICROCIRCUIT is cumulative with the preceding terms [emphasis added].” Had the delete functionality been implemented as described, there would be no question that ORing was cumulative since it would have been possible to delete the second to the last specifier in the search status (invalidating the “lock”).

Both the limited delete and the forced association of the OR were the focus of user criticism about the logic of the system. Users complained that A and B or C was always interpreted as A and (B or C), requiring (A and B) or C to be entered as (A or C) and (B or C). If A were the keyword “space” and B and C were authors, and the user wanted to specify articles on space written by either of two authors, the system gives the expected result by entering “Subject”, space, “Author”, Smith, "+", “Author”, Jones. But the designer intended this sequence of entries to isolate the articles Smith wrote about space together with all articles written by Jones, so that the designer would now continue with “Subject”, space to accomplish the goal of limiting the set to articles about space (the need to repeat the specifier indicates that the designer envisioned that a query involving either of two authors and one subject would be entered authors first, observing the pattern “expand then narrow”).

When the user wanted either articles on space written by Smith or articles on mars, the sequence “Subject”, space, “Author”, Smith, "+", “Subject”, mars isolated the expected articles by Smith but only those articles on mars that happened also to be indexed under space. The designer had intended the sequence to produce what the user wanted, but lack of communication, lack of quality programming time, or the inadequacies of the Univac 1050 resulted in a flawed implementation of Mitchell’s design. Note that the user should not begin with the references to mars and expect to OR in the articles by Smith on space because the AND implementation is never right associated (“Subject”, mars, "+", “Author”, Smith, “Subject”, space, "+",”Subject”, mars is required to isolate the desired set). Any design based upon incrementally manipulating a single working set will be restricted in the Boolean expressions it can generate and therefore will be more cumbersome to use than a system, such as Summit had designed, where multiple working sets are manipulated.

The second area of criticism highlighted in the evaluation concerned system response to the “Transmit” key. In his 1964 proposal Mitchell had predicted that searches would average ten question-answer sequences and require only three minutes of elapsed time to complete. Given light line loadings, he envisioned response times less than 3 seconds and service intervals about fifteen seconds (average time between presses of the “Transmit” key). However, he had assumed a dedicated computer that was limited by the speed of its disk controller (two disk accesses of 11 milliseconds each to answer each transmission).

The Univac 1050 became available in 1963 and was intended as “a general purpose subsystem employed mainly to supplement the parallel processing capabilities of UNIVAC III, UNIVAC 490 Real-Time and UNIVAC 1107 Thin Film Memory Computing systems” (Weik, 1964). The machine supporting the test had 16K of 4.5 microsecond RAM (United Airlines had been quoted a GE 635 with 1 microsecond memory six months earlier) and was attached to a Fastrand II Drum Storage System that held 132 million characters of data with an average access time of 92 milliseconds. The computer ran two partitions, foreground and background, so NASA shared cycles and memory with a program doing processing for a New York brokerage firm. This was clearly not the machine that Mitchell had envisioned to service the Direct Electronic Library, but the Teleregister Division had no other computer available and there was no funding to rent time on a faster computer. But neither were 100 concurrent searches being conducted. However, the estimates Mitchell had made as to disk accesses required to support a search were now low by an order of magnitude because the program was running off the drum, and the drum itself was slower by an order of magnitude, so the expected response time of below 3 seconds for 100 concurrent users became the expected response time for a single user.

Mitchell had initially anticipated that even if users knew exactly how to conduct their search they would spend most of the search duration keying input into their terminals. With the change in the search interface to minimize keying by presenting lists to users, the dialog between user and computer became heavily dependent on the computer’s ability to compute and transmit characters. He had foreseen that the heaviest load on the computer would be when users requested title display and so the user manual cautions “It is therefore recommend that you try to narrow your search terms as precisely as possible, to the point at which only a reasonable number of citations will be displayed for your viewing. This will be a convenience to you and a courtesy to others who may wish to use the RECON console” (Mitchell, 1966, 10). The evaluators made viewing a citation the criteria of a successful search and published the count of citations viewed, reporting that the average successful searcher pressed the “Title” key 9.3 times (only 39% of the attempted searches pressed the “Title” key).

Because of limited memory the Univac 1050 programs kept all user data on the drum, which meant that very few requests could be responded to in less than a second and that extensive inverted file manipulations involving creating a new working set on the drum would have taken several seconds. Under these circumstances the average response time was likely close to 3 seconds. If a search and display of results averaged 40 “Transmit” presses (due to increased interaction with lists), each averaging 3 seconds of response time, and if all users took an average of a 10 seconds to compose the next transmission (the terminal was locked until the computer responded), the average search would be about 9 minutes (this was the typical search time later reported by Summit (Lancaster, 1973, 192-196)). During the 10 second lull the computer could generate responses to 3.3 other users. Since the load on the system averaged about 180 searches a day during the trial, there would have been an average of 3.4 concurrent searches if the search distribution had been uniform, which it was not. It is likely that often during the trial there were more than 10 concurrent searches going on and it is not surprising that with likely ten to fifteen users sharing the processor the evaluators were able to measure mean delays not of 3 seconds, but of 38.5, 133.7 and 74.5 seconds on different searches (Meister, 1967, 15). It should have come as no surprise that as soon as more than 3 concurrent searches are attempted on this computer system, the response time would have begun to deteriorate. Since more than three concurrent searches was routine, many of the “failed” attempts and perhaps a good portion of the number of initiated searches can be attributed to bad-response-time frustration and repeated retries.

The third area of complaint was directed not at RECON but at what the users considered to be the poor quality of indexing. “RECON users were brought into very close personal contact with the characteristics of the present indexing system; this intimacy caused them to attribute some of their dissatisfaction with indexing terms to RECON” (Meister, 1967, 15). The evaluators analyzed system usage satisfaction separately for librarians and non-librarians but found the difference in their ratings of off-line and on-line searching not statistically significant even though the combined mean of the eight librarians rated on-line searching higher by .05 on a 7 point scale while the professional users rated on-line searching lower by .74. Librarians thought RECON more satisfactory and worthwhile than off-line searching but less useful and productive. Professional users found RECON considerably less satisfactory, but more worthwhile and useful and less productive than off-line searching. It seemed to the evaluators that librarians were annoyed by the inability to build sophisticated Boolean expressions to isolate citations while non-librarians were annoyed by the restrictive vocabulary that the system mandated, and everyone was annoyed by the slow response time. Wente, however, looked at the nearly 350 successful searches per week and concluded that “the test showed that working scientists and engineers both could and would use a hands-on retrieval terminal, and further, that they could obtain useful results directly with no intermediary personnel and only a minimum of instruction (Wente, 1971, 96). NASA was very pleased.

The project had cost Bunker-Ramo nearly $400,000. Just before the test began Dr. Ramo announced that he would step down as CEO and Milton Mohr took over as President and CEO. Mohr then suggested that Mitchell find a different job and gave him some introductions in TRW. At the end of the test NASA requested an extension at the same rental rate. Luther Harr, who was Executive VP, objected that he had several new brokerage customers ready to pay higher fees to use the Univac 1050. Mohr then told NASA that they would have to pay more than double “the rental prices on the consoles and communications gear involved, and NASA wouldn’t pay that much” (Mitchell, 1990, 151).

NASA now had to take a half step back. Fortunately, Melvin Day and Roger Summit had continued to converse about the on-line bibliographic retrieval system, now called DIALOG, that Lockheed continued to develop for in-house use on its new IBM 360/30. In June of 1966 NASA awarded a $20,000 contract to Lockheed to “install and operate a remote terminal at NASA Ames Research utilizing DIALOG to access the NASA file of 260,000 citations. The Ames Research Center is at Moffett Field in Mountain View, California, within 10 miles of the Lockheed facilities and DIALOG project staff ”(Bourne & Bellardo, in-press). It took Summit’s group until November to convert STI’s 1410 data files and build indices on the IBM 360 and then searching was instituted for 2 hours a day in January 1967. Wente states that “this contract award was based primarily on three factors: the need to test IBM’s then new Data Cell (2321) in on-line use; the availability of Lockheed’s independently developed computer programs at Palo Alto, and the real need at the Ames center nearby for a local capability to retrieve from NASA’s information collection" (Wente, 1971, 96). NASA had successfully hedged their bet on Bunker-Ramo by arranging a small-scale demonstration on state-of-the-art equipment and a proven search interface.
After a second RECON solicitation, this one based on NASA’s experience with both Bunker-Ramo and Lockheed, NASA awarded the second RECON contract to Lockheed in June of 1968.

In Retrospect

In 1966 Rowena Swanson asked the question about the repeated failure of information systems in the context of calling for better-planned systems that used the knowledge of technology experts as well librarians. Simon Ramo was the foremost expert in the practice of systems engineering and Bunker-Ramo specialized in real-time information systems. But Bunker-Ramo was driven out of the airline reservations business and walked away from the online information retrieval business, both for business reasons and business politics. It is clear that the Direct Electronic Library approach could have succeeded, if it had been properly implemented, because the systems analysis was sound. Because the Teleregister division was struggling to stay viable (it re-focused on the brokerage industry and in 1968 signed a contract to build the NASDAQ system) it did not have the spare resources to implement a new large-scale real-time project. A total of 8 employees worked on the RECON contract for different periods of time (not counting their support staff, computer operators, etc.), and none were assigned to it full-time. The project team was compartmentalized on either coast and not even Mitchell had good information about team performance and accomplishments. It is a testimony to the skill of the individuals in the two groups that the system came together on time and worked as well as it did.

In September of 1965 Herbert F. Mitchell wrote in his proposal to Goddard: The typical new electronic information-handling system cannot be designed best either by the customer alone, equipped though he may be to understand his own problems, or by an electronics hardware company that builds its customer approach around the conventional marketing of computers. It is necessary to come to understand simultaneously both the technology and the customer problem, and to envisage alternate ways of bringing a system of people and apparatus together to do the job better. The problem must be quantitatively assessed in detail, and economic-technological trade-offs for comparison of alternate approaches must be made (p. 6).

None of Summit, Borko, Kessler, Bourne, or Salton, took this approach to the online retrieval systems design problem. These men were closer to academia and simply started small, used the resources immediately available, and sought to make their prototypes useful to a local audience. These other pioneers enjoyed the time and freedom to tinker with their systems, experiment, revise, and discuss them with colleagues at professional meetings. Herbert F Mitchell, like Simon Ramo, simply solved the current problem and moved on to the next challenge.

What did the Direct Electronic Library leave behind? Meister and Sullivan published a single article and their contractor report. Those are the only records published by participants other than Herbert F. Mitchell’s privately published autobiography. Van Wente published one article from NASA’s perspective. Thus three articles have been the grist for scholars who have traced the development of online information retrieval systems. Summit and others copied Meister and Sullivan’s approach to evaluating online information systems simply because it was the first large-scale example. Sometimes the statistics published in the contractor’s report are mentioned (Wente quotes them) but there is no mention of the conclusions not published in their paper: “In addition to its significant performance in its primary role, RECON was successfully utilized to (1) update the very bibliographic collections upon which it is based, (2) trace the evolution of work in a given field, (3) provide a chronological record of an author’s work, and (4) trace the history of corporate involvement in a given field” (p. 13).

Lancaster and Fayen (1973) begin their review of Meister and Sullivan’s report with “Over a 7 week experimental period a staggering 6133 uses were recorded at six separate NASA centers” (p. 200). Other reviewers seem not to understand that no other online bibliographic retrieval system accumulated that number of uses (or that number of hours) for at least two more years (for the first 80 hour DIALOG test at Ames Lancaster cites Summit’s report that the system was down 24% of the scheduled time and in all 96 searches were completed, while in a subsequent 12 month trial at NASA headquarters only 13% of the 300 scheduled terminal hours were lost and approximately 300 searches were conducted (pp. 192-196)). Although less than 40% of RECON searches were considered successful, that still meant that in seven weeks nearly 2500 successful searches were conducted (we don’t remember Babe Ruth for setting the strikeout record)!

It is easy to critique the system from the point of view of information retrieval, which is what the users did. The NASA personnel wanted to find information (even though many were skeptical that what they were interested in what was in the library to begin with) and they expected that the system would lead them to the information they wanted (NASA did not present this as an experiment but as a new and improved tool). Salton and Borko were focused on the problem of automating the storage and retrieval of information, and they crafted their systems to help the user extract knowledge. Mitchell and Summit focused much more narrowly on the problem of retrieving documents via coordinate values. It was not their concern whether the indexing was accurate, what terms were in the thesaurus (although Mitchell kept track of terms usage so that synonyms could be increased), or whether the locus of the user’s input hinted at other pathways to the information he wanted to find.

Mitchell maintained that it was the purpose of the test to find out what users wanted in the system and how they wanted to use it, so his focus was simply to build an (inexpensive) system that did what the contract called for, provide large-scale remote access to the NASA citation collection with an easy-to-use interface. He would have welcomed the chance to refine the interface and the search strategy and had programmed the system to keep the usage data necessary to guide revisions (the source of Meister and Sullivan’s and of Wente’s statistics). Unfortunately, Bunker-Ramo made a business decision to abandon the program rather than continue the development of their investment. Like SRI in 1963, Bunker-Ramo was in the contracting business, not the information retrieval business.

In the computer field a successful software test is one that finds (correctable) errors. In recent times Wente reviewed 30 years of STI and presented a table of 8 major evaluations, ranging from 1967 to 1989 that helped the organization improve.

The management of any information program as large and complex as NASA’s scientific and technical information program justifies serious efforts at evaluating its effectiveness. Table 6 lists eight of the principal evaluations conducted by or for NASA in this area. Results of these studies were invariably useful, although specific changes made to the program in consequence are typically difficult to identify. However, the 1967 study on the 1966 testing of an online retrieval prototype clearly occupied a major role in issuance of the new NASA specification for an improved NASA/RECON system in 1968 (Wente, 1990).

The Direct Electronic Laboratory showed NASA what was possible and established a standard that NASA expected all future systems to meet. In May of 1967 Mitchell became the Director of the Operating Computing Branch at Goddard Space Center, and just before he retired four years latter he had the opportunity to use the DIALOG version of RECON. He easily understood the system and recognized many features that he thought (mistakenly) that he had originated. He never realized how much duplicative design work in online retrieval was going on outside his view during the two years he devoted to the Direct Electronic Library.


An Electronic Data Processing System for United Air Lines, Inc. (CP-227-1). (1965). Stamford, CN: The Bunker-Ramo Corporation, v1.

Borko, Harold. (1965). The Conceptual Foundationss of Information Systems(SP 2057), Santa Monica, CA: Systems Development Corp.

Borko, Harold & Burnaugh, H. P. (1966). Interactive Displays for Document Retrieval (SP 2557). Santa Monica, CA: Systems Development Corp.

Bourne, Charles P. (1999). 1999 Miles Conrad Memorial Lecture, NFAIS Annual Conference. Retrieved April 22, 2002 from

Bourne, Charles P. , & Bellardo, Trudi Hahn. (2002). A History of the Early Online Industry and Technology.

Brandhorst, W.T., & Eckert Philip F. (1966).Guide to the Processing, Storage, and Retrieval of Bibliographic Information at the NASA Scientific and Technical Information Facility(N66-34085), College Park, MD: Documentation Incorporated.

Drew, D. L., Summit, R. K., Tanaka, R. I., & Whiteley R. B. (1966). An On-Line Technical Library Reference Retrieval System. American Documentation, 17(1), 3-7.

Eklund, Jon (1994). The Reservisor Automated Airline Reservation System: Combining Communications and Computing. IEEE Annals of the History of Computing, 16 (1), 62-69.

Englebart, Douglas C. (1986). Workstation History and The Augmented Knowledge Workshop. Proceedings of the ACM Conference on the History of Personal Workstations. New York: ACM Press 87-100.

Lancaster, F.W. and E. G. Fayen. (1973). Information Retrieval On-Line, Los Angeles, CA: Melville Publishing Co.

Meister, David and Dennis J. Sullivan. (1967). Evaluation of User Reactions to a Prototype On-line Information Retrieval System, NASA CR-918. Canoga Park, CA: Bunker-Ramo Corporation.

Mitchell, Herbert F. (1964). Unsolicited Proposal, Direct Electronic Library, A study of the use of Electronic Library Reference Centers for the national Aeronautics and Space Administration, Bunker-Ramo Eastern Technical Center, Inc.

Mitchell, Herbert F. (1965a). Bunker-Ramo Direct Electronic Library, The Bunker-Ramo Corporation.

Mitchell, Herbert F. (1965b). Unsolicited Proposal for Technical Assistance to NASA Goddard Space Flight Center, (No. 5656-926). Silver Spring, MD: Bunker-Ramo Eastern Technical Center, Inc.

Mitchell, Herbert F. (1966). NASA/RECON — Computer/Library at Your Desk. Canoga Park, CA: Bunker-Ramo Corporation.

Mitchell, Herbert F. (1990). My Biography, Cape Coral, FL.

Ramo, Simon (1961). The Scientific Extension of the Human Intellect, Computers and Automation,10(2), 9-12.

Schatz, Bruce R. (1997, January 17). Information Retrieval in Digital Libraries: Bringing Search to the Net. Science, 275, 327-334.

Simpson, G. S. Jr., & Flanagan, C. (1966). Information Centers and Services. In Carlos A. Cuadra (Ed.) Annual Review of Information Science and Technology. New York: John Wiles & Sons. 305-335.

Sullivan, Dennis J., David Meister. (1967). Evaluation of User Reactions to a Prototype On-line Information Retrieval System. Proceedings of 30th Annual American Documentation Institute, v4, Thompson Books, 90-94.

Swanson, Rowena. (1967). Information System Networks — Let’s Profit from What we Know. In George Schecter (Ed.), Information Retrieval. Washington D.C.: Thompson Book Company.

Weik, Martin H. (1964). A Fourth Survey of Domestic Electronic Digital Computing Systems, REPORT N0. 1227 Aberdeen Proving Ground, Md: Ordnance Ballistic Research Laboratories Retrieved September 15, 2001 at

Wente, Van A. (1971). NASA/RECON and User Interface Considerations. In Donald Walker (Ed.) Interactive Bibliographic Search: the User/Computer Interface. Montvale, New Jersey: AFIPS Press.

Wente, Van A. (1990). Scientific and Technical Information Management. Government Information Quarterly, 7 (2), 149-167.

Return to Table of Contents.
Return to List of Appendices.