Issue 58 - Command Line Tools for Documentation Writers
Use deterministic CLI tools to search and update your docs.
If you work with a Docs-as-code workflow, you're working with automated tools and a publishing pipeline that transforms your text into other formats. It's helpful to understand how some of the tooling works, because that can help you build better scripts. In this issue, you'll explore a handful of commands you can use daily. And many of these commands are the ones coding assistants use under the hood to get things done.
CLI Commands for Your Docs Work
Writing is only part of the job when you work on documentation. You have to check that your site serves pages the way you expect, fix the same wording across hundreds of files when a product name changes, or audit pages for missing metadata.
You've probably pieced those jobs together with editor features and browser plugins. Lately, you might be tempted to hand them to an LLM instead. But ask an LLM to do the same job twice, and you might not get the same result both times.
Command-line tools don't have that problem. They're deterministic, which means they give you the same result every time. Issue 51 explains why LLMs aren't deterministic, and why that matters when you build workflows around them. And unlike an editor plugin, you can save a command in a script that your whole team runs, or run it in your build. CLI tools can work together, so you can combine several small tools to do bigger jobs.
They're also the tools coding agents run behind the scenes. When you know them, you can check an agent's work, or even build tools for your agents to use.
In this tutorial, you'll use five CLI tools that are built into macOS and Linux: curl, grep, sed, find, and awk. You'll use them to test a live website and then audit and make changes to a small set of local docs you'll create. Then you'll create a script you can use to replace text across your whole doc set. When you're done, you'll have five new commands you can use daily to speed up your work.
See what content a server sends with curl
Many docs sites offer a Markdown version of each page alongside the HTML, so tools and AI assistants can read the content without all the page markup. A server that supports this uses content negotiation: the client says which format it wants by sending an Accept header, and the server responds with the matching version. Use curl to watch it happen.
My site Small, Sharp Software Tools supports Markdown for each of the tutorials there. Run curl with the -I flag to fetch only the headers from the server's response:
$ curl -I https://smallsharpsoftwaretools.com/tutorials/monitor-with-updo/
The server responds with its headers:
HTTP/2 200
last-modified: Wed, 09 Sep 2026 17:04:21 GMT
etag: "4934-65b0fd7a7cf40"
accept-ranges: bytes
content-length: 18740
cache-control: max-age=600
expires: Sun, 27 Sep 2026 15:56:29 GMT
vary: Accept-Encoding,User-Agent,Accept
link: </tutorials/monitor-with-updo.md>; rel="alternate"; type="text/markdown"
link: </llms.txt>; rel="describedby"
x-llms-txt: /llms.txt
content-type: text/html
date: Sun, 27 Sep 2026 15:46:29 GMT
server: Apache
The content-type line confirms that the server returned HTML. The first link line indicates a Markdown version of this page exists and where to find it.
The vary line contains Accept, which tells caches that the response changes based on the format the client asks for. Without that, a cache or CDN could store the Markdown version and hand it to the next person who visits in a browser. If you run a docs site, make sure your server includes Accept in its Vary header.
Now ask for Markdown by using the -H flag to send an Accept header. Send the result to the head command with a pipe, the | character, so you only see the first five lines:
$ curl -s -H 'Accept: text/markdown' \
https://smallsharpsoftwaretools.com/tutorials/monitor-with-updo/ | head -n 5
The -s flag hides curl's progress output. This time, the server sends Markdown:
> ## Documentation Index
> Fetch the complete content index at: https://smallsharpsoftwaretools.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Monitor Your Web Apps with Updo
It's the same URL, but you got a different format. If your own docs site claims to support Markdown, this pair of commands is how you prove it's working.
curl shows you what a site sends. The rest of the commands work with files on your own machine, so you'll need some files to work on.
Set up the sample docs
To practice safely, you'll build a tiny docs folder with three Markdown files. Each file has front matter with a title, and two of them have a description. Several of them use the word "utilize" in different forms, which you'll clean up later.
First, create a project folder called text-tools and switch to it:
$ mkdir text-tools && cd text-tools
Next, create a docs folder:
$ mkdir docs
Then create the docs/index.md file:
$ cat <<'EOF' > docs/index.md
---
title: Get started
description: Install the scanner and run your first scan.
---
# Get started
This guide walks you through utilizing the scanner to check your site for broken links.
Utilize the install guide to set things up first.
EOF
The cat <<'EOF' line writes everything up to the closing EOF line into the file. The quotes around EOF tell your shell to leave the contents alone, so the dashes and backticks land in the file as written.
Next, create docs/install.md:
$ cat <<'EOF' > docs/install.md
---
title: Install the scanner
description: Install the scanner on macOS or Linux.
---
# Install the scanner
Most teams utilize a package manager to install the scanner.
To speed up large scans, install the scanner with GPU support:
```bash
scanner install --utilize-gpu
```
EOF
The install.md file has a code block with a command-line flag that contains the word "utilize." Remember that for later.
Finally, create docs/configure.md:
$ cat <<'EOF' > docs/configure.md
---
title: Configure scans
---
# Configure scans
The scanner utilizes a config file named `scan.toml`.
Set `max_depth` to control how deep a scan goes. Higher values mean more utilization of your network.
EOF
With your files in place, you're ready to search them.
Find what needs changing with grep
Before you change anything, you need to know every place the word shows up. The grep command searches files for text and prints the lines that match.
From your text-tools folder, use grep to search the docs folder for "utilize":
$ grep -rn utilize docs
The -r flag searches every file in the folder, and -n adds line numbers.
The command returns three matches:
docs/install.md:8:Most teams utilize a package manager to install the scanner.
docs/install.md:13:scanner install --utilize-gpu
docs/configure.md:7:The scanner utilizes a config file named `scan.toml`.
The grep command you used matches exact text, so it skipped "Utilize" with a capital letter, and it skipped "utilizing" and "utilization" because neither one contains the full string "utilize."
You can search with a pattern instead, because grep understands regular expressions. Square brackets match any one of the characters inside them, so [Uu] matches either a capital or a lowercase "u." Search for [Uu]tiliz, which catches every form of the word:
$ grep -rn '[Uu]tiliz' docs
Now the output returns six results:
docs/install.md:8:Most teams utilize a package manager to install the scanner.
docs/install.md:13:scanner install --utilize-gpu
docs/configure.md:7:The scanner utilizes a config file named `scan.toml`.
docs/configure.md:9:Set `max_depth` to control how deep a scan goes. Higher values mean more utilization of your network.
docs/index.md:8:This guide walks you through utilizing the scanner to check your site for broken links.
docs/index.md:10:Utilize the install guide to set things up first.
Look at line 13 of install.md. That's a flag in a code block in the documentation, and changing it would break the command, so you'll have to ignore that one later.
Make a habit of running grep before you change anything. Next, you'll use sed to change the word in a single file.
Change one file with sed
To change text without opening an editor, use sed. It applies your edits to each line of text and prints the result. By default, it doesn't touch the original file, which makes it safe to experiment with.
Run sed on docs/index.md to replace every form of "utilize" with the matching form of "use":
$ sed -E 's/([Uu])tiliz([ei])/\1s\2/g' docs/index.md
The expression s/find/replace/g tells sed what to change. The s means substitute. The first part is the pattern to find, and the second part is what to replace it with. The g at the end replaces every match on a line instead of only the first one. The -E flag turns on extended regular expressions, which work the same in the Linux and macOS versions of sed.
The pattern builds on the one you used with grep. Parentheses capture whatever they match so you can reuse it in the replacement. The first pair captures the "U" or "u." The second pair captures the letter right after "utiliz," which is either an "e" or an "i." In the replacement, \1 puts back the first captured letter and \2 puts back the second. So "Utilize" becomes "Use," and "utilizing" becomes "using."
The output shows both lines fixed:
---
title: Get started
description: Install the scanner and run your first scan.
---
# Get started
This guide walks you through using the scanner to check your site for broken links.
Use the install guide to set things up first.
The pattern leaves "utilization" alone, because an "a" follows "utiliz" in that word. That's on purpose. "Usation" isn't a word, and that sentence needs a person to rewrite it. If you want more reasons to hunt down "utilize," Issue 31 makes the case.
So far, sed has only printed its results. To save the changes back to the file, use the -i flag. This is where Linux and macOS differ. Most Linux distributions ship with GNU sed, and macOS ships with BSD sed, which expects a backup file extension after -i.
On Linux, run this command:
$ sed -E -i 's/([Uu])tiliz([ei])/\1s\2/g' docs/index.md
On macOS, pass an empty string after -i so sed doesn't create a backup file:
$ sed -E -i '' 's/([Uu])tiliz([ei])/\1s\2/g' docs/index.md
If you use the Linux version's syntax on a Mac, BSD sed treats the next argument as the backup extension, and you'll get an error or stray files you didn't expect. When you copy a sed command from the internet or from an agent's transcript, check which version it was written for.
You could run this command against your other two files, but if you had hundreds of files to edit, you'd want a better approach.
Change every file with find
To change every file at once, pair sed with find. The find command locates files that match rules you set, and it can run a command on every file it finds. You used it in Issue 12 to hunt for old files on your computer.
First, use find to look at the docs folder for all of the files with the .md extension:
$ find docs -name '*.md'
This shows all three files:
docs/install.md
docs/configure.md
docs/index.md
You can add the --exec flag to run a command on the resulting files.
First, preview what will change. Run find with --exec and use it to run sed. Pass the -n flag to sed, which stops it from printing every line, and add p to the end of the regex, which prints a line only when a substitution happens:
$ find docs -name '*.md' -exec sed -E -n 's/([Uu])tiliz([ei])/\1s\2/gp' {} +
Here's what all the options mean:
- The
-name '*.md'option matches every Markdown file. - The
-execoption runssedlike you ran it before, with the-noption to suppress output and the additionalpat the end of the regular expression to print when a substitution happens. findreplaces{}with the file names it found.- The
+at the end passes many file names to a singlesedcommand instead of runningsedonce per file.
Since you're doing a preview, you didn't use the -i flag to edit the original files, so the output shows each line as it would look after the change:
Most teams use a package manager to install the scanner.
scanner install --use-gpu
The scanner uses a config file named `scan.toml`.
The first and third lines look good. The second line would break the install command because --use-gpu isn't the flag the product uses. sed doesn't know what Markdown is, so it can't tell prose from code. It will make the same change every time you run it, which makes it predictable. It doesn't make it right, which is why you preview first.
You can tell sed to skip code blocks, but that makes the command longer, and you'd still need a Linux version and a macOS version to save the changes. It's time to take what you've built and turn it into a script.
Create a script for find and replace
Scripting is one of the biggest reasons to use command-line tools. A script saves the whole job in a file, so you write the tricky parts once, and after that you only need to remember the word you want to replace.
You'll create a script that uses find and sed to replace text in all of your content.
Create a file called replace.sh in your text-tools folder, next to the docs folder, and add the following code that defines the script:
#!/usr/bin/env bash
# Usage: ./replace.sh PATTERN REPLACEMENT [--apply]
pattern="$1"
replacement="$2"
skip='/^```/,/^```/!'
if [ "$3" = "--apply" ]; then
find docs -name '*.md' -exec sed -E -i.bak "${skip}s/${pattern}/${replacement}/g" {} +
find docs -name '*.md.bak' -delete
grep -rnE "$pattern" docs
else
find docs -name '*.md' -exec sed -E -n "${skip}s/${pattern}/${replacement}/gp" {} +
fi
The script takes a pattern, a replacement, and an optional --apply flag. Without --apply, it runs the same preview you ran by hand, so nothing changes until you're ready. With --apply, it makes the changes and then runs grep so you can see any matches that are left.
The skip variable holds an address. You can put an address in front of a sed expression to control which lines it touches. This is how you'll tell sed to ignore code blocks. The skip variable specifies a range that starts at a line beginning with three backticks and ends at the next one. The ! inverts the range, so sed changes every line except the ones inside code blocks. The script assumes every code block has a closing code fence, which is one more reason to keep your Markdown consistent.
When you pass --apply, the script calls sed with the -i.bak argument, which saves the original version of the file to a backup file. When you ran sed yourself previously, you needed one version of the -i flag for Linux and another for macOS. When you attach the backup extension directly to -i, with no space between them, both versions of sed understand it. The sed command saves a copy of each original file with .bak on the end, and then the script uses the find command's -delete option to clean those copies up. That makes the script portable.
Save the file and make it executable:
$ chmod +x replace.sh
The +x option marks the file as executable, so you can run it with ./replace.sh.
Run the script without --apply to preview the changes. Wrap the pattern and replacement in single quotes so your shell passes them through untouched:
$ ./replace.sh '([Uu])tiliz([ei])' '\1s\2'
The flag in the code block is no longer on the list:
Most teams use a package manager to install the scanner.
The scanner uses a config file named `scan.toml`.
The preview shows things look correct, so now run the script with the --apply flag to save the changes to the files:
$ ./replace.sh '([Uu])tiliz([ei])' '\1s\2' --apply
The script makes the changes and shows what's left:
docs/install.md:13:scanner install --utilize-gpu
The only match left is the flag inside the code block, which is what you want since it's a verbatim command option for the software you're documenting.
Run your grep search one more time to double-check everything got updated:
$ grep -rn '[Uu]tiliz' docs
The command shows the following output:
docs/install.md:13:scanner install --utilize-gpu
docs/configure.md:9:Set `max_depth` to control how deep a scan goes. Higher values mean more utilization of your network.
The "utilization" sentence needs a rewrite, and it's up to you to figure out how to handle that. The commands found the problem. Now you get to decide how to fix it.
If your docs live in a Git repository, run git diff before you commit so you can review every change the script made.
Check the script into your docs repository, and everyone on your team gets the same preview and the same changes, no matter which system they use.
Your prose is cleaned up. The last command turns your doc set into a report.
Audit your front matter with awk
To report on what's in your files, use awk. Like sed, it reads text one line at a time, but it's built for keeping track of things and making decisions as it reads.
With awk, you write small programs made of patterns, each followed by an action in curly braces. For every line it reads, awk checks each pattern and runs the action when the pattern matches.
In this case, you'll use awk to find every page that's missing a description field in its front matter. Search engines and other tools often use that field to summarize a page and display it in the search results.
Run this awk program against the files in the docs folder:
$ awk '
FNR == 1 { dashes = 0; found = 0 }
/^---$/ { dashes++; if (dashes == 2 && !found) print FILENAME; next }
dashes == 1 && /^description:/ { found = 1 }
' docs/*.md
The program has three patterns:
- The first pattern,
FNR == 1, matches the first line of each file.FNRholds the current line number within the file, so this resets two counters every timeawkstarts a new file. Thedashescounter tracks how many front matter fences you've seen, andfoundtracks whether you've seen a description. - The second pattern matches a line that contains only
---, which is a front matter fence. The action adds one todashes. When it reaches the second fence, the front matter is over, so iffoundis still zero,awkprintsFILENAME, which holds the name of the file it's reading. Thennextskips to the next line. - The third pattern matches a line that starts with
description:, but only whiledashesis one, meaningawkis still inside the front matter. When it matches, it setsfoundto one.
The output lists the one page without a description:
docs/configure.md
The docs/*.md pattern only matches files at the top level of the docs folder. To run this on a complete documentation set with multiple subdirectories, use find with the same awk program:
$ find docs -name '*.md' -exec awk '
FNR == 1 { dashes = 0; found = 0 }
/^---$/ { dashes++; if (dashes == 2 && !found) print FILENAME; next }
dashes == 1 && /^description:/ { found = 1 }
' {} +
You get the same result as the previous run, but now it works on every subfolder, and you have a list of pages to fix. Save it to a script like replace.sh so you don't have to type it again.
These five commands cover a lot of everyday docs work. You can check what your site serves, find every instance of a problem, fix it across hundreds of files, and report on what's left. For mechanical changes like these, run the command yourself.
Things To Explore
- pele is a lightweight library that renders Mermaid diagrams as SVG files using CSS variables.
- FreeCodeCamp has a hands-on Ethical Hacking Crash Course available. As people deploy more agents, it might be helpful for you to build some familiarity with how people attempt to gain access to systems.
Parting Thoughts
Before the next issue, try some of the following things:
-
Check your own docs site with
curl. Run the twocurlcommands against one of your pages. Does your platform offer a Markdown version? Does thevaryheader includeAccept? -
Find the external domains your docs link to most often. You'll need
grepwith its-oflag, which prints only the part of the line that matched, withsortanduniq -c.
You just read issue #58 of Code, Content, and Career with Brian Hogan. You can also browse the full archives of this newsletter.