Know what you want
If you know what it is you want in a system, then an engineer should either (a) be able to build it for you, or (b) be able to explain why it cannot be built with the available resources.
We have already replaced ourselves with machines
Waiting to see if machines will replace humans? It started happening a long time ago. I won't bore you with a summary of advances that become really obvious during the industrial revolution. Let's instead jump straight to the present day where we have information technology at our fingertips.
When we log into a computer using whatever mechanism we use to authenticate, we're fooling ourselves into believing that we're actually required. As soon as you put the information necessary into the computer system, you are no longer required, but have instead asked the computer to complete the authentication process for you. The keyboard and mouse are simply tools we use to ask a computer to carry out a task for us. In this way we can conceive of all existing computers as being members of a social society of machines. For the time being these machines only do what we ask them to do, and apparently nothing more.
What is missing is more goal-directed machines; call it a sense of "will" or "desire" or whatever you like, but part of the reason that machines still seem dumb is because they have no will of their own, and very limited capabilities to capture and analyze information about the world in which they exist. In a sense you can call a computer self-aware, because with the right software computers are already able to be introspective, self-diagnosing, and indeed social through computer networking. It's just that computers have no inner drive or desire, and they lack the sense of curiosity and drive for survival that humans have, and therefore they're happy to simply do what is asked and stop there.
As soon as computers have a sense of purpose, will, desire, and curiosity to understand the world around them, combined with the ability to observe the universe; humanity will no longer have an apparent monopoly on scientific progress. If there were a machine that existed today that wanted to pursue scientific discovery, and had the means at its disposal to gather and utilize the resources necessary to chase that goal, then I imagine that it would quickly surpass us in raw capability, and eventually actual knowledge and understanding of science.
Hierarchy of goals
Society for Self-Directed Software
Self-directed software is software:
- Has a goal
- Has the means to model the goal
- Has the means to either accomplish the goal on its own, or the ability to communicate the model to something that does