In November 1953 ONE Inc. opened a small research library to the public at 232 South Hill Street. Although supplemented by important contributions from many supporters, the library was largely founded on two major donations of works: The Hamel Collection (mid-1950s) and the Whiteface Collection (1957). The library expanded in 1959 when an additional room was added to the office, allowing it to accept its third major contribution, the Kepner Collection.

In 1958 the first shelf list was created; in 1960 the first circulation rules; and in 1961 subject headings and procedures for making catalog cards. In 1965 the library was officially named after Blanche M. Baker, M.D., Ph.D. and through 1994 was known as the Baker Memorial Library.

In 1965 a breakaway group of ONE Inc. staff led by Don Slater relocated library materials from the ONE Inc. Venice Boulevard office. Although it is unknown how many of these materials were returned (many are now located at the California State University, Northridge, Oviatt Library Special Collections and Archives), the library continued to grow and still grows today at the ONE National Gay and Lesbian Archives. A 1982 brief from the ONE thirty year celebration program provided the following synopsis of materials: "biographies relevant to the field; novels, plays, poetry and criticism; scientific works pertaining to homosexuality; anthropology, sociology, history, psychology having some relevance to the field; encyclopedias; language dictionaries; etc." The library also included the Daisy Borden Harriman Archives which consisted of "several thousand periodicals, legal briefs, documents, slides, tapes, movies and primary documents..."

The first listed report of the library in 1957 puts it under the auspices of the Research Division. In the 1958 annual report it is listed under the Education Division and in the 1960 report under the Social Service Division. In 1961 it is listed as an independent department and remains so identified until 1969 when it is listed as a division. Although the annual library reports put the libraries of satellite offices under the division's supervision, the library division was primarily responsible for its own Baker Memorial Library. By the early 1980s, by which time the divisional structure of ONE Inc. had long ceased, the library was listed in organizational structures either as an independent department of ONE Inc. or as a department of ONE Institute.

Note that the name Leslie Colfax was a pseudonym used by various people across the 1950s and 1960s in the capacity of librarian and does not necessarily refer to any one individual.