Is the "Signifier" Arbitrary?

Anyone interested in linguistics will know that for Ferdinand Saussure, the father of modern structuralism, post-structuralism and semiotics, there existed a "signifier" and "signified": the "signifier" was the linguistic term or phrase used to convey meaning and the "signified" was the object referred to. However, that is not quite true. Saussure bracketed the referent (or "thing-in-itself") and declared that the "signifier" referred not to something in the real world, but to a concept in our minds. This is to say, that for Saussure, the linguistic phrase was representative of our "idea" of reality--but not necessarily reality as it really is. For example, we know very well what a corner is--but where does a corner begin and end? Our perception tells us one thing, but possibly the reality is something else. For these reasons of verification, Saussure avoided saying that the signified is an object in the real world: on the contrary, the signified is only a concept in our own minds. The signifier, for Saussure was "arbitrary", in the sense that there was no necessary connection between the linguistic phrase and the thing or idea represented. For example, "a house" could just as easily be called "a shoe" as long as everyone accepted the new meaning ("A rose by any other name would smell as sweet" as Shakespeare puts it). In his "Tractatus-Logico-Philosophicus" Ludwig Wittgenstein proposes the idea that language has a common logic with reality. Admittedly, he eventually renounced this point of view--but is there any way in which Saussure's theories about epistemology could be interpreted as supporting this early Wittgenstinian idea? It would certainly seem that the theories involved are mutually exclusive, as if language is "arbitrary" there can be no common logic between it and reality. However, the earlier point about Saussure "bracketing the referent" becomes vital here. If the signifier does not refer to a real thing, but to a concept in our minds, then it may be possible to assume that though language does not share a common logic with reality, it does have a common logic with our perception of reality (which might be constructed from Chomsky's "universal grammar"?). Furthermore, if our perception of reality portrays a more or less "true reality" then we might reasonably say that language shares its logic with reality. Of course, the objection

would be: "To what extent does our perception of reality reflect reality as it really is?" Personally, I believe that all living creatures perceive the world more or less as it truly is: this is necessary for survival in an often hostile environment. Was Wittgenstein possibly wrong to discard his early model?