Many words have more than just a single meanings. The determination of the intended meaning from its context is a skill learnt early on in language development. For example, differentiating set as referring to a bone healing, as opposed to a film stage, is straightforward. However, when a word has two or more meanings which are similar enough to fit into the same context, then we can encounter problems.
The main meaning of famous is well-known, but it can also mean first-rate. Thus, if an unknown Harry Smith is described as famous, we can't rely on language structure forcing only one meaning to fit. Instead, we depend on other clues being given. For example, if he was only mentioned in a footnote as a famous author, we would have to assume one of the meanings. Alternatively, if he was described in a paragraph lauding his achievements, then the intended meaning would be obvious.
Infamous, literally not-famous has only the single definition, that of having a bad reputation. This allows the possibility that the description of famous and infamous can be applied to the same person. However, the strangeness of this mitigates its use. 'Adolf Hitler, famous and infamous leader ...' anyone?
[As a rather confusing aside, famous also has an archaic meaning of ill repute.]
Notorious is a good, solid synonym for infamous. However, it is also a synonym for widely known, or famous.
Known widely : famous, notorious
Known widely and esteemed : famous
Known widely and unfavorably: infamous, notorious
Now that that's all cleared up, you can get to see a whole world of confusion out in the real world. The first result on Google for "famous and infamous" is currently The Great, the Famous and the Infamous. Here, if famous meant 'known widely' then 'great' and 'infamous' are superfluous. If it meant 'known widely and esteemed', then 'great' is superfluous. Indeed, the whole first page of google results will give you a headache. Wasn't life simpler when you didn't think about this stuff?