As algorithms, such as trading algorithms, are just one kind of empirical theories, the question is: Can we trust tests of empirical theories? According to theory of science, the answer is, strictly speaking, no.

All empirical theorists, a.k.a scientists, can do is look at the past, try to find patterns and regularities in the past, and predict the future based on the found patterns and regularities. However, while more often than not the future behaves like the past, there is no such thing as an obligation for the future to behave like the past. We might have observed thousands of squirrels, we might have offered tuna to thousands of squirrels, and not one squirrel might have eaten the tuna. That doesn't mean that we will never find a squirrel that actually eats tuna. That is why, according to theory of science, we can never actually verify empirical theories. All we can do is try to disprove them. And if a theory has withstood many attempts to disprove it, then it can be considered to be a good theory.

The same principle applies to backtesting trading strategies. No amount of data about the past will be sufficient to predict the future with certainty. The future can always be different from the past. The dollar may always have behaved in a certain way. But then there are changes in geopolitics and, suddenly, the dollar behaves in a different way.

Does that mean that trading is pointless?

Of course not. Then science would also be pointless. And we see every day that it is not.

The past is all we have. We do not have data about the future. We can use data about the past to predict the future. We just need to be aware of the fact that, in the realm of empirical theories (as opposed to mathematics and logic), there will never be absolute certainty.

Now, if we want to invent trading strategies and test them: How should we do that? Obviously, we need to look at a lot of data. We need to do many tests. We need to modify trading strategies and test them again, if they do not work. If, for example, the "buy gold after three days of falling prices" strategy does not work, we might want to try "buy gold after four days of falling prices". Obviously, this is a lot of work for humans. So we should use computers. What we can do is write programs that do this work for us. Such programs are often called "bots".