Modifying robots.txt for individual sites of a multisite install

WordPress creates a robots.txt dynamically. To overwrite it in a normal non-multisite installation, you can just upload a static robots.txt to the server. On a multisite install, this would overwrite the robots.txt for all sites, which is not always the wanted behavior. This post explains how you can modify robots.txt for individual sites of a multisite.

WordPress comes with the filter robots_txt which allows modifying the dynamically created robots.txt’s output. The function get_current_blog_id() returns the ID of the current multisite site, which we can use to check for a particular site to add rules to the robots.txt. This is how it looks currently for my site:

/** * Modify robots.txt for main site and english site * * @param $output * @param $public * * @return string */ function fbn_custom_robots( $output, $public ) { $site_id = get_current_blog_id(); if ( $site_id == 1 ) { $output .= "Disallow: /agb-und-widerruf/n"; $output .= "Disallow: /mein-konto/n"; $output .= "Disallow: /warenkorb/n"; $output .= "Disallow: /impressum-und-datenschutz/n"; } elseif ( $site_id == 11 ) { $output .= "Disallow: /account/n"; $output .= "Disallow: /cart/n"; $output .= "Disallow: /imprint/n"; $output .= "Disallow: /terms/n"; } return $output; } add_filter( 'robots_txt', 'fbn_custom_robots', 20, 2 );

For the site with the ID 1 (this is are added four Disallow rules, likewise for the site with the ID 11 (my English site

To get the site’s ID, I just added the following line after $site_id = get_current_blog_id();:

$output .= $site_id;

This way, the ID of the current site is displayed, when you visit its robotx.txt.

6 comments on »Modifying robots.txt for individual sites of a multisite install«

    1. Florian Brinkmann

      Hi Adam,

      sorry for the late reply, your comment was marked as spam…
      To your question: yes, you can modify the default’s robots.txt file with that filter.


  1. Matt R

    Hi Florian,

    Thank you for this solution. Can you please tell me what this line does? I'm not an expert in PHP.

    add_filter( 'robots_txt', 'fbn_custom_robots', 20, 2 );

    Thank you!

      1. Matt R

        Thanks Florian,

        Where by chance would you add this code in a multisite setup? Is there a centralized file that would handle this for each site?

        1. Florian Brinkmann

          Hi Matt,

          you’re welcome.

          I use it in a custom plugin that is enabled network-wide. I pasted the code into a (very basic) plugin file that you can use as a starting point:

          You could use it as a Must Use plugin ( For that, just upload it to the wp-content/mu-plugins folder via FTP (you may need to create the mu-plugins directory).

          Hope that helps,


Leave a Comment

Your email address will not be published. Required fields are marked *