Modifying robots.txt for individual sites of a multisite install

WordPress creates a robots.txt dynamically. To overwrite it in a normal non-multisite installation, you can just upload a static robots.txt to the server. On a multisite install, this would overwrite the robots.txt for all sites, which is not always the wanted behavior. This post explains how you can modify robots.txt for individual sites of a multisite.

WordPress comes with the filter robots_txt which allows modifying the dynamically created robots.txt’s output. The function get_current_blog_id() returns the ID of the current multisite site, which we can use to check for a particular site to add rules to the robots.txt. This is how it looks currently for my site:

/**
 * Modify robots.txt for main site and english site
 *
 * @param $output
 * @param $public
 *
 * @return string
 */
function fbn_custom_robots( $output, $public ) {
	$site_id = get_current_blog_id();
	if ( $site_id == 1 ) {
		$output .= "Disallow: /agb-und-widerruf/n";
		$output .= "Disallow: /mein-konto/n";
		$output .= "Disallow: /warenkorb/n";
		$output .= "Disallow: /impressum-und-datenschutz/n";
	} elseif ( $site_id == 11 ) {
		$output .= "Disallow: /account/n";
		$output .= "Disallow: /cart/n";
		$output .= "Disallow: /imprint/n";
		$output .= "Disallow: /terms/n";
	}

	return $output;
}

add_filter( 'robots_txt', 'fbn_custom_robots', 20, 2 );

For the site with the ID 1 (this is florianbrinkmann.com) are added four Disallow rules, likewise for the site with the ID 11 (my English site florianbrinkmann.com/en).

To get the site’s ID, I just added the following line after $site_id = get_current_blog_id();:

$output .= $site_id;

This way, the ID of the current site is displayed, when you visit its robotx.txt.