John Quincy Adams
In 1804, Federalist Senator Timothy Pickering (1745-1829) called for a constitutional amendment apportioning each state's representation in the House of Representatives solely on the basis of the number of freemen. Such an amendment would have overturned the Three-Fifths Compromise and greatly reduced the number of slave state representatives.
While Federalists, during the first years of the nineteenth century, attacked the three-fifths clause as a source of Republican power, they hesitated to directly challenge the institution of slavery itself. Their descendants, however, would assume a leading role in the antislavery campaign. Nevertheless, it is striking that as early as 1804, Adams was already thinking in terms of a "Slaveholding power."
...I have long thought it an important error, of many good and distinguished men among us, that they are too ready to indulge that love of ease and domestic comfort--Too ready to withdraw from the field of public action.... This love of retirement and domestic pleasures, has in this state kept in confinement to their chimney corners, numbers of men, who ought to stand forth the guardians of the public interest, and the guides of our public opinions--It is the only thing which can possibly hazard the steadiness of our politics--I hope they will even resist the dangers of this drowsy opiate....
You will have seen by the proceedings in our Legislature that a serious alarm has of late been so active at the seat of government to establish an impregnable rampart of Slaveholding power, under the false batteries of democracy--The Senators of the State in Congress are instructed to propose and endeavour to effect an amendment of the Constitution so that the representation in the National House of Representatives may be a representation of freeman--I believe this alteration must be made, and I have no doubt it will be effected whenever those states can be United in its favour.... If this great majority of the numbers, wealth, and strength of the Country can be made to harmonize in the pursuit of an object so obviously just in itself, and so clearly important to them I cannot doubt but they will obtain it--This however must be the work of Time and of chance perhaps more than of anything else.