The Library was founded in 1856 by about a dozen students of the Elphinstone College living in the Fort Area in Bombay (now Mumbai) and was named the ‘Fort Improvement Library’. Ten years later in 1866 it was renamed the ‘Fort Reading Room and Library’ and its members numbered about 250. In 1891 the late Mr. Nesserwanjee Manockji Petit gave a donation of Rs. 25,000/- with the intention of perpetuating the memory of his deceased and only son the late Mr. Jamsetjee Nesserwanjee Petit who for many years was one of the Life members and a director of the library and had during his lifetime taken a very lively interest in its welfare and progress. The name of the library was then changed to ‘The J. N. Petit Fort Reading Room and Library’. At that time the library was located in a big hall rented for the purpose on the first floor of a house in Churchgate Street.

About 1895 Bai Dinbai Nesserwanjee Petit donated a sum of about Rupees two and a half lakhs for a building to be erected in memory of her son late Mr. Jamsetjee Nesserwanjee Petit. The new building, in the present location, consisting of two storeys and one mezzanine floors, was inaugurated on 1st May, 1898. A set of entirely new and elegant furniture for the library was presented to the Institute by Bai Avanbai Framji Petit, sister of the late Mr. Jamsetjee Nesserwanjee Petit. Sir Dinshaw Manockjee Petit, 1st Baronet, gave a donation of Rs. 10,000/- to form the nucleus of the Book Purchase Fund. Bai Dinbai N. Petit also donated the entire private Library of Mr. Jamsetjee Nesserwanjee Petit consisting of more than 500 volumes, several of which were very rare and costly works. The extension of the building was undertaken in 1938, and the extended premises were declared open for the use of members on the 1st May 1939 by Sir Dinshaw Manockjee Petit, 3rd Baronet, the President of the managing Committee of the library at that time.

The building was given the Urban Heritage Award for the year 1990 by the Bombay Heritage Society.