1
0
mirror of https://github.com/jbranchaud/til synced 2026-07-13 02:16:09 +00:00

Add Track Line Occurrences From Input With AWK as a Unix TIL

This commit is contained in:
jbranchaud
2026-07-12 17:44:54 -05:00
parent b520d6525b
commit 895fefdc58
2 changed files with 55 additions and 1 deletions
+2 -1
View File
@@ -10,7 +10,7 @@ working across different projects via [VisualMode](https://www.visualmode.dev/).
For a steady stream of TILs, [sign up for my newsletter](https://visualmode.kit.com/newsletter).
_1818 TILs and counting..._
_1819 TILs and counting..._
See some of the other learning resources I work on:
@@ -1859,6 +1859,7 @@ If you've learned something here, support my efforts writing daily TILs by
- [Switch Versions of a Brew Formula](unix/switch-versions-of-a-brew-formula.md)
- [Tell direnv To Load The Env File](unix/tell-direnv-to-load-the-env-file.md)
- [Touch Access And Modify Times Individually](unix/touch-access-and-modify-times-individually.md)
- [Track Line Occurrences From Input With AWK](unix/track-line-occurrences-from-input-with-awk.md)
- [Transform Text To Lowercase](unix/transform-text-to-lowercase.md)
- [Type Fewer Paths With Brace Expansion](unix/type-fewer-paths-with-brace-expansion.md)
- [Undo Changes Made To Current Terminal Prompt](unix/undo-changes-made-to-current-terminal-prompt.md)
@@ -0,0 +1,53 @@
# Track Line Occurrences From Input With AWK
In [Deduplicate List While Preserving Original
Order](deduplicate-list-while-preserving-original-order.md), I showed a terse
AWK pattern that allows for sifting out all duplicate lines as they are
encountered. This looks like `!seen[$0]++`.
I thought it would be useful to look at a less dense version of this where I
break out the conditional check, make the `print` explicit, and add lines to the
associative array in the action block.
```bash
echo "red green blue red yellow green blue red green" | tr ' ' '\n' | awk '!($0 in seen) { print; seen[$0] = 1 }'
red
green
blue
yellow
```
Let's take a look at that. The first part is the pattern that determines whether
the action(s) runs.
```bash
!($0 in seen)
```
Here we check if the current line from the input being processed (`$0`) has
already been added to the associative array we declared with a name of `seen`.
If it is the first time we've seen that exact line, then it won't be in, so a
`false` which gets negated (`!`) to `true`, so the actions are triggered.
The second part in curly braces is a sequence of actions separated by
semicolons.
```bash
{ print; seen[$0] = 1 }
```
The first action is `print` which will print the current line to stdout. The
second action adds the current line to the associative array (`seen`) with a
value of `1`. Now any time we encounter a recurring line it will be present _in_
`seen` and the pattern will evaluate to false, preventing these actions from
running.
The whole thing then is:
```bash
awk '!($0 in seen) { print; seen[$0] = 1 }'
```
Again this is an expanded, easier-to-understand version of `awk '!seen[$0]++`
which has the same behavior.