Can the lithium test be applied to differentiate brown dwarfs from other substellar objects? The lithium test is often used to differentiate brown dwarfs from low-mass M-type dwarfs (see e.g. Martin et al. (1994)), because brown dwarfs (at least the lower-mass ones) do not burn lithium, whereas M-type dwarfs can and do.
Can the lithium test be applied to distinguish brown dwarfs from other substellar objects, such as massive gas giants or sub-brown dwarfs? I don't know what the lithium abundances are in those objects, though I expect it would depend on the exact type. Such a test could work if brown dwarfs have comparatively higher lithium abundances, though these other objects would not fuse lithium, either.
 A: The lithium test has an ambiguity because it also depends on the age of the objects.
Li is depleted once the cores of fully convective objects reach temperatures of about $3\times 10^{6}$ K. The time at which this occurs depends on the contraction timescale of newly formed objects. This timescale is longer for lower mass objects.
Thus we can say that objects with $M<0.06 M_{\odot}$ will never burn their lithium. However, lithium will also survive in younger objects with higher masses.
For instance, in the Pleiades, with an age of 125 Myr, Li is expected to survive in all objects with $M<0.075M_{\odot}$, hence it is an ideal test to determine whether an object is a brown dwarf or not in clusters at that age.
In younger clusters, the mass limit is even higher. For instance at 35 Myr, objects with $M<0.12M_{\odot}$ will keep their lithium (Chabrier & Baraffe 1997).
In fact, this dependence is now used to estimate the age of young clusters by finding the lowest luminosity (mass) stars which still retain all their lithium - known as the "lithium depletion boundary" age (Jeffries & Oliveira 2005).
The lithium test was successfully applied to identify brown dwarfs in the Pleiades by Basri et al. (1996) and Stauffer et al. (1998), and establish the cluster age as 125 Myr, older than previously thought (the older age is why earlier efforts to find Li failed in what turned out to be low-mass stars rather than brown dwarfs).
The Li-test cannot be used to discriminate between brown dwarfs and even lower mass objects. We expect such objects to retain their cosmic Li abundance (currently $A(Li) = 3.3$ on a logarithmic number density scale where the hydrogen abundance $A(H)=12$), so that all objects with $M<0.06M_{\odot}$ should have their full complement. A further problem with using Li is that it will start to form molecules in the atmospheres of very cool, low-mass objects (e.g. LiH) making interpretation extremely difficult.
However, an exciting avenue for research is the "deuterium test". Deuterium behaves like Li, but at lower masses. In other words, D-burning takes place at temperatures below $10^{6}$ K and occurs at the centres of extremely low mass brown dwarfs ($\geq 0.013M_{\odot}$) on mass-dependent timescales of a few to 50 Myr. The detection of D in the atmosphere of an object would give you an upper limit to its mass, which depended on its age. i.e If it were an old object, then D-detection would tell you it had $M<0.013M_{\odot}$. The mass limit would be higher at younger ages (Chabrier et al. 2000).
The detection of deuterium will require the detection of HDO or CrD molecules in the mid-IR part of the spectrum, but will possibly be feasible with the James Webb space telescope. More importantly than merely identifying objects below a certain mass, the "deuterium depletion boundary" might offer them means to get the absolute ages of young clusters in the crucial range 2-20 Myr, when planetary systems are forming (Soderblom et al. 2014).
