Hi Sascha,
as far as I know you cannot specify different color sets for text/background in a way that makes the core blocks like paragraph use them. The colors defined in add_theme_support( 'editor-color-palette' );
are used for all color controls by default.
Best,
Florian