The founders did a trick on racist slavery

They didn't really mention it, except other people. In the free states, involuntary servitude was going away for whites, black slaves freed are freed to what? A presumption that legislatures cannot make labor law on race. So, an agreement  with the slave states just becomes reflected in naturally evolving labor rights, in general. Slavery drops away as a long lost custom, never reflected in Constitutional Law. If we just had CAA back then. We are racist pricks, government makes slaves; do not mix the two.