Personally I believe if you think something Joseph Smith taught contradicts the Book of Mormon, you are believing in propaganda from your church/group.

2 Nephi (LDS 29:8 – 9) (RLDS 12:58-63)
8 Wherefore murmur ye, because that ye shall receive more of my word? Know ye not that the testimony of two nations is a witness unto you that I am God, that I remember one nation like unto another? Wherefore, I SPEAK THE SAME WORDS UNTO ONE NATION LIKE UNTO ANOTHER. And when the two nations shall run together the testimony of the two nations shall run together also. 9 And I DO THIS THAT I MAY PROVE unto many that I AM THE SAME YESTERDAY, TODAY, AND FOREVER; and that I speak forth my words according to mine own pleasure. And because that I have spoken one word ye need not suppose that I cannot speak another; for my work is not yet finished; neither shall it be until the end of man, neither from that time henceforth and forever.

Alma (LDS 7:20) (RLDS 5:34-35) I perceive that it has been made known unto you, by the testimony of his word, that he cannot walk in crooked paths; NEITHER DOTH HE VARY FROM THAT WHICH HE HATH SAID; neither hath he a shadow of turning from the right to the left, or from that which is right to that which is wrong; therefore, his course is one eternal round.

Now taking it for granted that the scriptures say what they mean, and mean what they say, we have sufficient grounds to go on and prove from the Bible that the gospel has always been the same, and the officers to officiate, the same; and the signs and fruits resulting from the promises, the same… – Joseph Smith, Teachings of the Prophet Joseph Smith, page 264

If any man preaches to you, doctrines contrary to the Bible, the Book of Mormon, or the Book of Doctrine & Covenants, set him down as an imposter…Try them by the principles contained in the acknowledged word of God; if they preach, or teach, or practice contrary to that, disfellowship them; cut them off from among you as useless and dangerous branches. – Joseph Smith, Times & Seasons (maybe vol 5 page 490)

A quick note about the Doctrine and Covenants in the quote above. I use the 1835 Doctrine and Covenants, the last edition to be printed before Joseph Smith murder. And here is what it has to say about polygamy: It states one man and one wife, to marry in public where everyone can see it, and that polygamy is a crime, summary of 1835 D&C 101. FYI the only major church that removed it was the LDS church. See more at my Marriag resource page, including Joseph Smith modified journal entry not long before he was murdered which originally spoke aganist polygamy.