From dd13203ade04e370c690da0946bd9d9dda4e239d Mon Sep 17 00:00:00 2001 From: jbranchaud Date: Tue, 5 Dec 2023 16:28:05 -0600 Subject: [PATCH] Add Progress Reporting To Long Running Script as a Ruby TIL --- README.md | 3 +- ...ogress-reporting-to-long-running-script.md | 46 +++++++++++++++++++ 2 files changed, 48 insertions(+), 1 deletion(-) create mode 100644 ruby/add-progress-reporting-to-long-running-script.md diff --git a/README.md b/README.md index 4f59230..83e26ff 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ pairing with smart people at Hashrocket. For a steady stream of TILs, [sign up for my newsletter](https://crafty-builder-6996.ck.page/e169c61186). -_1355 TILs and counting..._ +_1356 TILs and counting..._ --- @@ -1045,6 +1045,7 @@ _1355 TILs and counting..._ - [A Shorthand For Rerunning Failed Tests With RSpec](ruby/a-shorthand-for-rerunning-failed-tests-with-rspec.md) - [Add Comments To Regex With Free-Spacing](ruby/add-comments-to-regex-with-free-spacing.md) - [Add Linux As A Bundler Platform](ruby/add-linux-as-a-bundler-platform.md) +- [Add Progress Reporting To Long-Running Script](ruby/add-progress-reporting-to-long-running-script.md) - [Are They All True?](ruby/are-they-all-true.md) - [Assert About An Object's Attributes With RSpec](ruby/assert-about-an-objects-attributes-with-rspec.md) - [Assoc For Hashes](ruby/assoc-for-hashes.md) diff --git a/ruby/add-progress-reporting-to-long-running-script.md b/ruby/add-progress-reporting-to-long-running-script.md new file mode 100644 index 0000000..7ea1ae6 --- /dev/null +++ b/ruby/add-progress-reporting-to-long-running-script.md @@ -0,0 +1,46 @@ +# Add Progress Reporting To Long-Running Script + +A script that is going to iterate over and process a ton of data can take a +long time to run. It can be hard to predict how long it will take to run. So, +it is nice to have some indication of it's progress as it goes. + +Here is a small snippet that I add to the main loop so that I have some +feedback while the script runs: + +```ruby +running_count = 0 + +tons_of_data.each do |item| + running_count += 1 + if running_count % 1000 == 0 + print '@' + elsif running_count % 100 == 0 + print '*' + else + print '.' + end + + # process this `item` ... +end +``` + +Then I get some output that looks like this over time as the script runs. + +``` +............................................................................... +....................*.......................................................... +.........................................*..................................... +..............................................................*................ +............................................................................... +....*.......................................................................... +.........................*..................................................... +..............................................*................................ +...................................................................*........... +............................................................................... +.........*..................................................................... +..............................*................................................ +...................................................@.................. +``` + +I can get a sense of how quickly it is processing each record and roughly count +how far along it is.