Not sure what kind of complexity you mean, but I can make a guess. There are different kinds of complexity metrics. The one I encountered the most was the "depth of loops", which is obviously not meant here.
Let me give you an example. You might have an audio set of devices. They are wired to each other, each to the wall socket (the electrical system of your house), each to the amplifier, which connects to the loudspeakers. This is a fairly normal design.
Some people connect the grounding connection to a water pipe. Water pipes are grounded, so this looks like a good deal. However, this makes the system overly complex. Your system now also depends on the water system, which can lead to unwanted consequences. Your audio system can suffer from ground loops if it is also grounded through the electrical system, and if a device fails, you could get electricity on your tap water. Ouch!
So, from a design point of view, there should not be a dependency to the water system at all when there is no water involved. Not even if the water system is conveniently close.
I think your complexity mainly deals with "lone ranger" objects that try to everything themselves, and therefore have knowledge of too many objects outside their field of responsibility.
If you distribute the responsibilities, you can introduce levels of abstraction that makes the design more flexible and more maintainable. In you audio system, the sound plugs are a good example of such abstraction. Your amplifier does not care if you connect a cassette player, a DVD player, an MP3 player, etc. A bad example is the gramophone plug, as this refuses to implement the standard "audio plug" interface by giving the wrong signal strength. So each amplifier has to deal with them separately, which makes the system overly complex.
Does this answer your question?
+++ Despite being wrong in every important aspect, that is a very good analogy +++
Hex (in Darwin's Watch)