While I appreciate the orientation towards an analytical, scientific and health perspective here, I have not ignored culinary considerations where they seemed important.
This post is a work in progress. I will try to keep updating it as I learn more.
General theory
Beans, legumes, grains, nuts and, uh, seeds are all the seeds of the plant they come from. They have presumably evolved to store nutrients in durable forms and resist fully breaking down in the digestive systems of creatures that might swallow them so they may reach suitable soil and germinate. These foods thus have surfaces that physically and/or chemically resist digestion and protect the contents.
My impression is that the full picture is somewhat more complex, and not widely understood. The majority of available information on the topic is poorly referenced, sometimes contradictory, and not always evidence-based. Some writers argue that we should not be eating these foods at all, others recommend soaking, sprouting and other techniques, and others write that such measures are unnecessary.
While some of the substances in these foods that may be affected (or not) by soaking them aren't useful, others may have nutritional benefits. I would advise readers to consider their particular dietary requirements and keep an open mind and an eye out for credible research about this area of food and nutrition science.
Beans
The web is full of articles urging us to soak beans or not to soak beans. Finding any actual data on this topic is hampered by the noise of contradictory anecdotes and vague statements.
Considering the few scraps of evidence derived from scientific approaches (here I broadly mean actually measuring things and controlling conditions) I was able to find, it seems that soaking beans has the same benefit as boiling them, in terms of reducing levels of oligosaccarides such as raffinose and stachyose that cause flatulence due to their fermentation by bacteria in the large intestine. Soaking and boiling may reduce the levels of these chemicals more than boiling alone. It should be noted that oligosaccarides are prebiotics, meaning they are used by beneficial gut bacteria, the importance to health and complexity of which is, I think, only beginning to be understood...
Some beans also contain toxic lectins. Not all lectins are harmful (and even the toxic ones have various scientific and medical uses), but red kidney and cannelini beans, for example, contain a lectin (soy beans contain another) that causes severe food poisoning if the beans are not cooked. Lectins can also act as "antinutrients", binding nutrient chemicals so that they cannot be absorbed. Soaking at high temperature may reduce levels of lectins, but they need to be strongly heated to be fully destroyed (they are proteins, and so they denature at high temperature) and long soaking and low temperature cooking may increase toxicity. Ten minutes of vigorous boiling is recommended.
This study for example shows that levels of raffinose, stachyose and lectins decline during hot soaking, and then decline further during cooking. Unfortunately, it does not attempt to determine whether the benefits of soaking could be achieved by a cooking process, such as cooking for longer, cooking at higher pressure or temperature and so on. Note that the soaking temperature used was high; 77 degrees C at the start, then 24 degrees C.
This study found that soaking reduced the levels of gas-producing oligosaccarides in 5 types of beans (including Bengal gram/kala chana which is a type of chickpea). Soaking with a very small quantity of bicarbonate of soda reduced levels significantly further (this can cause the skins of beans to break down further than they otherwise would. I make mushy peas by soaking dried green peas with bicarbonate of soda before cooking). Germination (sprouting) for 24 hours reduced levels further. After a 48 hour germination period, the oligosaccarides were undetectable.
While anecdotal evidence says everything possible, I've read a few articles stating that soaking beans for longer than usual, say 24 hours, resulted in the eater suffering less digestive symptoms such as flatulence than short soaking, and my own experience and my family's bears this out. The temperature seems influential; beans definitely sprout faster if the temperature is higher, and whatever activities start to break down the surface chemicals seem to proceed faster in warmth. When the indoor temperature is less than 20 Celsius, I aim to soak beans for ~20 hours. When the indoor temperature is around 23-25 Celsius, 8 hours seems to do the trick. If it's warmer than that, I soak beans in the fridge (for up to 48 hours), as they start fermenting after about 12 hours otherwise.
Beans also contain fairly high levels of phytic acid, which binds zinc, iron and calcium so that they cannot be absorbed. This article which refers to some studies on the topic, states that soaking decreases phytic acid content in beans. However, unlike the oligosaccarides that break down inside the beans during soaking and cooking, losing their discomfort-causing properties, phytic acid for the most part merely leaches into the water, and the soaking water must be discarded to obtain reduced levels in the food. Soaking in an acidified solution (add a few drops of lemon juice) is more effective than plain water. For most uses, I prefer to cook beans in the soaking water, because I think they taste better, but if you are concerned about reducing phytates, it may be better to discard. Sprouting beans (and lentils) causes the phytic acid (which, among other uses, is a phosphate store for the seed) to break down due to enzyme activity, and of course, sprouting involves rinsing off the soaking water. Note that phytates are thought by some to be nutritionally beneficial (and apparently have anti-cancer properties) and may not cause any adverse symptoms besides reduced mineral absorption (which may be a big problem for some people).
I'd like to note that soaking reduces the cooking time of beans (this is an environmental consideration: less energy is required to cook them, and if there were no other benefit of soaking, I personally would recommend it on that basis).
Sprouting beans
Some beans (and lentils, and seeds like mustard and alfalfa) (whole mung and moth beans are excellent for this, and chickpeas also do well) can be soaked, rinsed, and left to sprout (they need to be rinsed regularly to hydrate and remove any potentially harmful microorganisms) for 12-48 hours depending on the ambient temperature. These beans can be eaten raw (though this may be recommended only for immunocompetent people) or cooked. As mentioned earlier, studies show that germination effectively destroys gas-producing chemicals and phytic acid.
Lentils
Like beans, lentils contain phytic acid. This can be removed by soaking and draining the soaking water, but lentils are usually not soaked or soaked only briefly (and probably for culinary purposes, as I think is the case with rice).
Lentils also contain resistant starch which cannot be broken down into glucose by the human body, and is instead fermented by bacteria in the large intestine like those oligosaccarides we met earlier. This produces fatty acids which are absorbed and metabolized, and... gases in the gut. Soaking does not destroy resistant starch. Rather, levels are reduced by other processes such as grinding, or cooking in excess water. However, resistant starch reduces the glycemic index/load of the food (this means it reduces the overall rise in blood glucose caused by eating it), suggesting that it may be beneficial in improving or reducing the risk of developing insulin resistance, metabolic syndrome and type 2 diabetes, so eliminating it may not be desirable (as with phytates and oligosaccarides).
Nuts and seeds
I need to find more data for this section.
Nuts and seeds generally have much higher levels of phytic acid (which binds to iron, calcium and zinc, preventing their absorption) than cereal grains and lentils, but since they are usually eaten in small quantities this is not necessarily a problem in a balanced diet. Where nuts and seeds are eaten in large quantities, reducing phytates may be a good idea.
Phytic acid content of nuts and seeds can be reduced by roasting or toasting, but soaking in a slightly acidic solution (add a few drops of lemon) overnight followed by thorough rinsing and dehydration (otherwise the food will decay) is more effective. I really have to mention here that the difference in taste between soaked nuts and unsoaked nuts is noticeable. Soaked and dried nuts and seeds taste as good or better to me as toasted nuts. This suggests to me that the process makes some good tasting (and possibly nutritious) chemicals more available, but I need to find more evidence for this.
The effort and energy required to dehydrate nuts and seeds after soaking is too much for me (they can be bought, but they are prohibitively expensive), so as a compromise I might either toast/roast them if they need to be stored for a few days or used dry, or just soak and rinse if they can be used wet (for example, in raw desserts or cashew cream).
Summary Table
Substance toxic lectins raffinose & phytic acid resistant starch
stachyose
Why remove? food poisoning cause gas binds Zn/Fe/Ca causes gas
Sources some beans beans beans, lentils
grains,
lentils,
nuts & seeds
Removed by no reduced reduced no
soaking?
Drain soak - no yes ?
water?
Better soak - baking soda acid -
Alternative high heat soak+sprout soak+sprout grind/boil
removal fermentation fermentation
Why not - lower GI possible lowers GI
remove? prebiotic benefits prebiotic
Note that baking soda is alkaline so you cannot simultaneously soak in an acidified solution and a baking soda solution (you could use excess acid, but the soda would be destroyed, having formed a sodium salt with whatever anion the acid provided). Thus, if you want to enhance your soak, you have to choose whether to remove more oligosaccarides or more phytic acid.
