In the 1840s, large numbers of immigrants came to America; many Irishmen settled on the eastern seaboard and Germans moved out to Midwest farmlands. Numerous newcomers were Roman Catholics. The majority became associated with the Democratic Party. Opposition developed quickly. Eastern factory workers feared job losses to immigrants who were often willing to work for very low wages. Others feared the newcomers simply because they were different – in looks, language, customs and religion. These fears led to the growth of "nativism," a belief that only native-born or long-established citizens should have a voice in public affairs. The Irish famine greatly increased immigration from that country between 1845 and 1850, and correspondingly raised the fears, often irrational, of the existing population. By the late 1840s, secret anti-immigrant organizations began to form in a number of states. They used different names, but collectively they were referred to as the “Know-Nothings.” This moniker, first employed by the New York Tribune on November 16, 1853, arose from the members’ reluctance to talk about their organizations: When asked about their activities they would often say, “I know nothing.” One of the most famous of these groups was the Order of the Star Spangled Banner in New York State; others used similarly high-sounding names. New York (1844) and Boston (1845) elected nativist mayors and the 29th Congress include four nativist members, four from New York and two from Pennsylvania. A national convention was held in 1845 under the banner of the "Native American Party." Modern students will find this name curious, since it consisted entirely of whites of European descent, precisely the group now excluded by the term "Native American." Neither then nor now is the term used to include everyone who is a native American. In 1854, an effort was made to expand the movement through the formation of the American Party. The organization advocated a 21-year residency requirement for citizenship and the limitation of public office to native-born Americans. Members actually went farther and promised to support only Protestants. State houses throughout the country were captured by the Know-Nothings and governors were elected in Massachusetts and Delaware. The Know-Nothings victory in Massachusetts was nothing short of sensational, including the governor, all members of Congress and all but three members of the Massachusetts legislature. However, the arrest of Anthony Burns under the Fugitive Slave Act provided the incoming governor, Henry Gardner, a controversy he could not avoid. The judge who sent Burns back into slavery was Edward G. Loring of Suffolk County. Twice Bills of Address were sent to Gardner, urging him to remove Loring but he refused. It remained for his Republican successor to do the deed. Slavery was to be the party`s undoing. At their national convention in Philadelphia in 1855, the southern representatives pushed through a platform supporting slavery, anti-slavery elements led by Henry Wilson of Massachusetts, walked out. Although there were some electoral victories yet to come, the party`s peak of influence had been reached. In the Election of 1856, the American Party split over the slavery issue. Candidate Millard Fillmore ran poorly and was able to carry only the state of Maryland. Governor Gardner of Massachusetts, who won re-election to the governship, not surprisingly attributed the party`s failure at the national level to immigrants:
While this horde of foreign-born voters has thus stricken down a noble cause, which appealed to the moral sentiment and enlightened patriotism of our country, it only affords another confirmation of a fact which our whole history establishes, that the foreign vote, with hardly an exception, always has been and, in the nature of things, ever will and must be attracted to that party which, under high-sounding generalities on the abstract rights of man, always practically cooperates with slavery at the South, and banishes from its platform the moral questions and nobler instincts and more enlightened sentiments of the age.The party quickly became an object of ridicule and was entirely abandoned in its region of origin, New England. Most of the records that detailed the party`s early origins were destroyed, so that historians now have difficult determining for certain the facts about the period.