What is the key to intelligent systems?
The key is trust. If we cannot trust an intelligent system, then we will not let it take control. In fact, it is probably true for all kinds of systems—especially software intensive systems. If we cannot trust a system, then there will be trouble and trouble comes in many forms: political, economic, deployment, operational control, administrative, change management, velocity, productivity, quality, perception, customer confidence, etc. You get the idea.
How can you make a software intensive system more trustworthy?
You can do it with tests and behaviors. Not only should you provide automated tests for development, but automated tests for production as well. And once you have automated tests, you can provide automated behaviors.
If you do not have any tests at all, then I bet you have trouble instead.
Great design needs to be trustworthy. What do you think?
1 comments:
I agree that intelligent system have to be trustworthy. Otherwise it is useless, since every decision it makes should be verified by expert (who could make the decision himself).
Personally I wouldn't be satisfied just by automated tests. Those are only ensure that in predefined conditions (mostly by the same developer who made the system) system behaves properly.
Audit and timely notification along with extended period of real situation testing adds a lot.