Buy
Collected Fictions and
Selected Non-Fictions (Penguin, 1998 and 1999).You can really start anywhere and not go wrong (although most people advise that
Universal History of Infamy, which is first in
Collected Fictions is not JLB's best work or particularly similar to his other stuff.) Personal favorites stories of mine are "Tl?n, Uqbar, Orbis Tertius," "The Approach to Al-Mu'tasim," "Pierre Menard, Author of the
Quixote," "The Lottery of Babylon," "The Garden of the Forking Paths," "Funes, His Memory," "The Cult of the Phoenix," and "On Exactitude in Science." Also the essay "A New Refutation of Time."
I should add that you can find many of his stories and essays (including "http://www.acsu.buffalo.edu/~jatill/175/libraryf.htm," which I'd guess is one of the most widely read) online if you don't want to spend any money.