Terminating a computer program and eating a fish, which is more cruel
Yesterday, when I was walking my dog in the park, I saw a guy fishing. It was raning a little bit, but he seems pretty confortable with it. He caught a fish, checked it very quickly and threw it back to the river. Looks the fish was too small. Suddenly I remembered that I heard that fish had only seven seconds of memory. Well, it's kind of reasonable, it is possible because the oxygen level in the water is too low, how matter how good fish gills cal filter oxygen, it's not enough to support a large brain. But in fact, the memory of fish is much better than many people think(1)Suddenly I recall reading the article some time ago, Nvidia got the largest NLP model in history, MegatronLM, saying that it has 8.3 billion parameters. So the question is: are there many models that are now more developed than the brains of some simple creatures?
So I want to first check out some simple creatures. Since the IQ of fish may be much higher than we think, let’s start with invertebrates. For example a Caenorhabditis elegans, each of which contains exactly 302 neurons. (2) So forget MegatronLM, your Pytorch project is probably more complicated than its "brain". I want to find some smarter creatures as the MegatronLM level, but if I continue to search like this, I might need the rest of the century. Then I found another article, there is a list on Wikipedia The number of animal nerve cells. (3) . In this way, MegatronLM is more complicated than the brains of many animals. It even reaches the complexity of the Grizzlies! But the problem is that in this list, elephants have more neurons than human neurons. So what is this about?
First of all, elephants are really smart, their IQ can reach 4-5 years old children. (4) So the number of neurons can still determine many things, perhaps because elephants don’t have fingers, so they can’t train their brains better, or elephants are too big too strong to have enough natural enemies in Africa, so they are not motivated to evolve? In any case, more neurons indeed mean cleverness. I think the correlation between the two should be somehow a linear regression problem. So is MegatronLM already smarter than the Grizzlies?
First, how to define smartness. Narrow sense of intelligence has been defined as how to get higher scores in IQ tests. Of course, some people will find it more important to have better innovation capabilities. So it seems smart is not as well defined as we think.
So, is the Grizzlies smart? In the natural world, especially in the traditional living area of grizzly bears, forest etc., he must be smart, he is even smarter than many people. Because he knows how to use his claws and how to use the forest to catch prey. If a person is not well trained, even if he has got machine gun, he may not be able to survive in the forest because he does not know how to hide himself before approaching his prey. However, if we change the environment, the grizzly may not be so smart. In the modern city's concrete forest, the grizzly bear's IQ may be extremely low because he does not know how to survive in this environment. (Of course someone might catch it in the zoo, and then carefree for a lifetime)
Ok, some people may think of it here. Isn't human beings similar to animals? Many immigrants left their home and headed for a foreign country will feel uncomfortable and homesick. The biggest problem is the language problem. So if you see this unfitness to the environment as a more abstract idea, language may be the most important part of the environment for human beings.
What about MegatronLM? It can train/learn the BERT model in 53 minutes. (5) . As for what the BERT is, here is a simple demo (6) . Can the Grizzlies do this after 53 minutes of their birth? Certainly not, not even human being. Of course, you can't say that MegatronLM has a better ability to learn semantic analysis, I mean it has no self-awareness that it is reading/analyzing a paper. However, it is obviously smarter in this special "environment".
So now the question is: Next time you start training a model that is complicated enough, you use a template (code stored on the hard disk) to some extent to create a "creature" (in RAM) that might be at least more complex than invertebrates. So, when you press Ctrl+C to terminate the program, is it equivalent to killing a fish?