mirror of
https://github.com/jbranchaud/til
synced 2026-07-13 02:16:09 +00:00
Add Track Line Occurrences From Input With AWK as a Unix TIL
This commit is contained in:
@@ -10,7 +10,7 @@ working across different projects via [VisualMode](https://www.visualmode.dev/).
|
|||||||
|
|
||||||
For a steady stream of TILs, [sign up for my newsletter](https://visualmode.kit.com/newsletter).
|
For a steady stream of TILs, [sign up for my newsletter](https://visualmode.kit.com/newsletter).
|
||||||
|
|
||||||
_1818 TILs and counting..._
|
_1819 TILs and counting..._
|
||||||
|
|
||||||
See some of the other learning resources I work on:
|
See some of the other learning resources I work on:
|
||||||
|
|
||||||
@@ -1859,6 +1859,7 @@ If you've learned something here, support my efforts writing daily TILs by
|
|||||||
- [Switch Versions of a Brew Formula](unix/switch-versions-of-a-brew-formula.md)
|
- [Switch Versions of a Brew Formula](unix/switch-versions-of-a-brew-formula.md)
|
||||||
- [Tell direnv To Load The Env File](unix/tell-direnv-to-load-the-env-file.md)
|
- [Tell direnv To Load The Env File](unix/tell-direnv-to-load-the-env-file.md)
|
||||||
- [Touch Access And Modify Times Individually](unix/touch-access-and-modify-times-individually.md)
|
- [Touch Access And Modify Times Individually](unix/touch-access-and-modify-times-individually.md)
|
||||||
|
- [Track Line Occurrences From Input With AWK](unix/track-line-occurrences-from-input-with-awk.md)
|
||||||
- [Transform Text To Lowercase](unix/transform-text-to-lowercase.md)
|
- [Transform Text To Lowercase](unix/transform-text-to-lowercase.md)
|
||||||
- [Type Fewer Paths With Brace Expansion](unix/type-fewer-paths-with-brace-expansion.md)
|
- [Type Fewer Paths With Brace Expansion](unix/type-fewer-paths-with-brace-expansion.md)
|
||||||
- [Undo Changes Made To Current Terminal Prompt](unix/undo-changes-made-to-current-terminal-prompt.md)
|
- [Undo Changes Made To Current Terminal Prompt](unix/undo-changes-made-to-current-terminal-prompt.md)
|
||||||
|
|||||||
@@ -0,0 +1,53 @@
|
|||||||
|
# Track Line Occurrences From Input With AWK
|
||||||
|
|
||||||
|
In [Deduplicate List While Preserving Original
|
||||||
|
Order](deduplicate-list-while-preserving-original-order.md), I showed a terse
|
||||||
|
AWK pattern that allows for sifting out all duplicate lines as they are
|
||||||
|
encountered. This looks like `!seen[$0]++`.
|
||||||
|
|
||||||
|
I thought it would be useful to look at a less dense version of this where I
|
||||||
|
break out the conditional check, make the `print` explicit, and add lines to the
|
||||||
|
associative array in the action block.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
❯ echo "red green blue red yellow green blue red green" | tr ' ' '\n' | awk '!($0 in seen) { print; seen[$0] = 1 }'
|
||||||
|
red
|
||||||
|
green
|
||||||
|
blue
|
||||||
|
yellow
|
||||||
|
```
|
||||||
|
|
||||||
|
Let's take a look at that. The first part is the pattern that determines whether
|
||||||
|
the action(s) runs.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
!($0 in seen)
|
||||||
|
```
|
||||||
|
|
||||||
|
Here we check if the current line from the input being processed (`$0`) has
|
||||||
|
already been added to the associative array we declared with a name of `seen`.
|
||||||
|
If it is the first time we've seen that exact line, then it won't be in, so a
|
||||||
|
`false` which gets negated (`!`) to `true`, so the actions are triggered.
|
||||||
|
|
||||||
|
|
||||||
|
The second part in curly braces is a sequence of actions separated by
|
||||||
|
semicolons.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
{ print; seen[$0] = 1 }
|
||||||
|
```
|
||||||
|
|
||||||
|
The first action is `print` which will print the current line to stdout. The
|
||||||
|
second action adds the current line to the associative array (`seen`) with a
|
||||||
|
value of `1`. Now any time we encounter a recurring line it will be present _in_
|
||||||
|
`seen` and the pattern will evaluate to false, preventing these actions from
|
||||||
|
running.
|
||||||
|
|
||||||
|
The whole thing then is:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
awk '!($0 in seen) { print; seen[$0] = 1 }'
|
||||||
|
```
|
||||||
|
|
||||||
|
Again this is an expanded, easier-to-understand version of `awk '!seen[$0]++`
|
||||||
|
which has the same behavior.
|
||||||
Reference in New Issue
Block a user