From 1264425124d796d78b06222bb6ce65dca782abea Mon Sep 17 00:00:00 2001 From: jbranchaud Date: Sun, 24 Jan 2021 17:17:38 -0600 Subject: [PATCH] Add Load Records In Batches With find_each as a rails til --- README.md | 3 +- .../load-records-in-batches-with-find-each.md | 37 +++++++++++++++++++ 2 files changed, 39 insertions(+), 1 deletion(-) create mode 100644 rails/load-records-in-batches-with-find-each.md diff --git a/README.md b/README.md index dfe2fc6..6630c81 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ pairing with smart people at Hashrocket. For a steady stream of TILs, [sign up for my newsletter](https://tinyletter.com/jbranchaud). -_1023 TILs and counting..._ +_1024 TILs and counting..._ --- @@ -646,6 +646,7 @@ _1023 TILs and counting..._ - [Inspect Previous Changes To ActiveRecord Object](rails/inspect-previous-changes-to-activerecord-object.md) - [List All Installable Rails Versions](rails/list-all-installable-rails-versions.md) - [List The Enqueued Jobs](rails/list-the-enqueued-jobs.md) +- [Load Records In Batches With find_each](rails/load-records-in-batches-with-find-each.md) - [Log SQL Queries Executed By ActiveRecord](rails/log-sql-queries-executed-by-activerecord.md) - [Mark A Migration As Irreversible](rails/mark-a-migration-as-irreversible.md) - [Make ActionMailer Synchronous In Test](rails/make-action-mailer-synchronous-in-test.md) diff --git a/rails/load-records-in-batches-with-find-each.md b/rails/load-records-in-batches-with-find-each.md new file mode 100644 index 0000000..bfe7d29 --- /dev/null +++ b/rails/load-records-in-batches-with-find-each.md @@ -0,0 +1,37 @@ +# Load Records In Batches With find_each + +The base enumerable method offered by Ruby is `#each`. If you need to interact +with an array of elements, that's a method you'll reach for at some point. + +When working with an `ActiveRecord` collection in Rails, you should use the +[`#find_each`](https://api.rubyonrails.org/v6.1.0/classes/ActiveRecord/Batches.html#method-i-find_each) +method instead of `#each`. That's because under the hood it batches the records +that it will load in 1000 at a time. This is important to keep your server's +resource usage from exploding when requesting a ton of records. + +Consider a `users` table that contains 10,000 records that are _active_. + +```ruby +User.where(active: true).each do |user| + # do something +end +``` + +With `#each`, all 10,000 records will be loaded into memory at once as +`ActiveRecord` objects. That's potentially a lot of load on the server's +available memory. Then imagine the table contains 100,000 or 1,000,000 records. +This can become a big problem. + +```ruby +User.where(active: true).find_each do |user| + # do something +end +``` + +With `#find_each`, which uses +[`#find_in_batches`](https://api.rubyonrails.org/v6.1.0/classes/ActiveRecord/Batches.html#method-i-find_in_batches) +under the hood, only 1000 `ActiveRecord` objects get loaded into memory at a +time. + +If you want to exercise more control over the batching, you can use +`#find_in_batches` directly.