How to Efficiently Load and Process 10,000 Nodes in Drupal Without Killing Performance

If you need to retrieve data from 10,000 nodes in Drupal, the most efficient way is to load them in batches, not one-by-one in a loop and not all at once.

✅ Recommended Approach:

1. Use entityQuery to get the node IDs:

$nids = \Drupal::entityQuery('node')
  ->condition('status', 1)
  ->range(0, 10000) // Limit to 10,000 nodes
  ->execute();

2. Load nodes in chunks (e.g., 500 at a time):

$batch_size = 500;
$nids = array_values($nids); // Reset keys to use array_slice

for ($i = 0; $i < count($nids); $i += $batch_size) {
  $chunk = array_slice($nids, $i, $batch_size);
  $nodes = \Drupal::entityTypeManager()->getStorage('node')->loadMultiple($chunk);

  foreach ($nodes as $node) {
    $title = $node->label();
    $field_value = $node->get('field_example')->value;
    // Process the data
  }
}

🚫 Avoid These:

• Don't load one node per loop — very slow and inefficient:

  foreach ($nids as $nid) {
    $node = \Drupal\node\Entity\Node::load($nid); // ❌ bad performance
  }
  

• Don’t load all 10,000 at once:

  $nodes = \Drupal::entityTypeManager()->getStorage('node')->loadMultiple($nids); // ⚠️ may cause memory issues
  

💡 Extra Tips:

• If you only need a few fields (like title or a custom field), it’s faster to use the Database API directly rather than loading full node entities.
• For long processes started via UI, consider using Drupal’s Batch API to avoid timeouts.
• Cache the result if this is a repeated operation.

To get the node titles of 10,000 nodes efficiently in Drupal, you have two main options:

✅ Option 1: Use entityQuery + loadMultiple() (safe batch loading)

use Drupal\node\Entity\Node;

$nids = \Drupal::entityQuery('node')
  ->condition('status', 1)
  ->range(0, 10000)
  ->execute();

$nids = array_values($nids); // Re-index array
$batch_size = 500;

for ($i = 0; $i < count($nids); $i += $batch_size) {
  $chunk = array_slice($nids, $i, $batch_size);
  $nodes = \Drupal::entityTypeManager()->getStorage('node')->loadMultiple($chunk);

  foreach ($nodes as $node) {
    $title = $node->label();
    \Drupal::logger('my_module')->notice("Node title: $title");
  }
}

✅ Option 2: Use raw SQL (faster, if you only need nid and title)

$query = \Drupal::database()->select('node_field_data', 'n');
$query->fields('n', ['nid', 'title']);
$query->condition('status', 1);
$query->range(0, 10000);

$result = $query->execute();

foreach ($result as $record) {
  $title = $record->title;
  \Drupal::logger('my_module')->notice("Node title: $title");
}

• ✅ Much faster than loading full node entities.
• ❗ Only works if you're using the default title field (not renamed or customized).
• ❗ Does not give access to other fields or translations unless explicitly joined.

💡 Summary: