For nine weeks a job on my laptop read Google Search Console every Monday and told me exactly three things to fix. In week one it found a page with 349 impressions and a 0.57% click-through rate, rewrote the title, and logged the change as a win. Seven weeks later the same query had 812 impressions and a 0.12% click-through rate. The job still had it filed under “keep doing this”.
This is the write-up of that job: what it looked at, what it recommended, what actually happened to the numbers, and the two design mistakes that made it look busy while it learned nothing. If you are building any weekly “recommend → ship → learn” loop (SEO, ads, email subject lines), the mistakes transfer directly.
The setup in one paragraph
The site sells mobility products in Mexico, plus a glucose monitor we distribute (I’ll call it the CGM cluster: a product guide, a buying guide and a few supporting pages). The job pulls the last 28 days of Search Console data for that cluster, finds candidate problems, scores them, and writes a one-page memo capped at three actions. Each action has a type (title/meta, new content, internal links, schema), an effort tag, the evidence row that triggered it, and an Approve / Dig deeper / Skip choice. Nothing deploys until I approve. The Search Console API is free, so the whole loop costs $0 a week.
What the memo actually looked like
This is the first memo, from late July, lightly trimmed:
CGM cluster (28d): 2 clicks · 349 impressions · CTR 0.57%
Strongest query: "qitrax" (the product's brand name) — position 7.3
A1 New content Effort L Publish a México-focused page for
"monitor continuo de glucosa"
Evidence: no Mexican query row with real volume yet
A2 Title/meta Effort S Rewrite the guide's title + description
Evidence: 349 impressions at pos 7.3, CTR 0.57%
A3 Schema Effort S Product + FAQ structured data,
distributor + sanitary registration lines
I approved all three. All three shipped within the week. That is the part that worked: three concrete changes, each small enough to preview, each tied to a row of evidence. Compared with the 140-item audit exports I used to ignore, it was a real improvement.
What happened to the numbers
Here is the same cluster, week by week, as the job reported it (28-day rolling windows):
| Week | Cluster clicks | Cluster impressions | Brand query position | Brand query CTR |
|---|---|---|---|---|
| Jul 22 | 2 | 349 | 7.3 | 0.57% |
| Jul 29 | 3 | 619 | 7.8 | 0.32% |
| Aug 5 | 6 | 967 | — | — |
| Sep 2 | 9 | 519 | 9.1 | 0.21% |
| Sep 9 | 9 | 706 | 8.2 | 0.33% |
| Sep 16 | 8 | 1,005 | 7.8 | 0.12% |
Read it as an outsider would. Impressions roughly tripled. Clicks went from 2 to about 8. On the one query the title fix targeted, the click-through rate fell to a fifth of where it started. The cluster grew because other pages started appearing for other queries, not because the fix worked.
Mistake 1: the verdict meant “shipped”, not “worked”
The job kept a learning file it re-read before every memo, split into “Do more of” and “Avoid”. All three July actions sat under “Do more of” from the first week onward. Look at what the outcome field said for the title fix: “Live title: Monitor continuo de glucosa en México…”. That is proof the change was deployed. It is not a result.
The bug is easy to write and hard to notice, because every individual piece looks reasonable: the action was approved, it shipped, someone filled in the outcome with what they could verify that day. The “Avoid” list stayed empty for two months, and an empty avoid list should have been the alarm. A recommender that never finds a mistake is not learning; it is just recording what it did.
The fix is to separate three things that the original job collapsed into one field: the baseline (numbers for that exact page and query the day it ships), the checkpoints (the same numbers at fixed intervals afterwards), and the verdict, which may only be computed from the first two. A minimal version:
def verdict(baseline, checkpoint, min_impressions=200):
# Not enough data yet: never call it a win by default.
if checkpoint["impressions"] < min_impressions:
return "waiting"
before = baseline["clicks"] / max(baseline["impressions"], 1)
after = checkpoint["clicks"] / max(checkpoint["impressions"], 1)
if after >= before * 1.2:
return "keep"
if after <= before * 0.8:
return "revert_or_rethink"
return "flat"
Run against the table above, the title fix gets “revert_or_rethink” by the second checkpoint. That single line would have saved a month of reapproving a change that was not doing anything.
Mistake 2: three weeks of the same three actions
In September the memos for three consecutive weeks proposed the same three actions: rewrite the brand query’s title, publish a Mexico-focused page, add schema. All three had already shipped in July. The diagnosis rules still matched (low CTR on a high-impression query, no strong Mexican head-term row), and nothing in the job knew those exact fixes had been tried.
A weekly recommender needs a memory of attempts, not just of findings. Before an action goes into the memo, check whether the same type of change already shipped on the same page within a cooling-off window. If it did, the memo should say so and either wait for the verdict or propose something genuinely different. Repeating a recommendation is the fastest way to train the reader to stop reading.
Why the title fix could not work
With hindsight the data was telling a simpler story. “qitrax” is a brand name. People searching for a brand mostly want the brand’s own site, and we are a distributor sitting around position 8. Impressions rose because more people learned the name, partly from our own ads, but a better title on the eighth result does not beat the manufacturer’s page for a navigational search. My best explanation for the CTR drop is that those extra searchers were even more navigational than the early ones. Either way, the job could not tell the difference between “the snippet is bad” and “this query is not ours to win”.
The rule I would add: classify queries before scoring them. Brand queries for brands you do not own get reported, but they cannot generate a CTR action. The effort goes to category queries (“monitor continuo de glucosa”, “andaderas para adultos”) where position and snippet quality actually decide the click.
What I changed
In October the job was rebuilt around a change tracker. Every recommendation becomes a row with the page, the target queries, the change type and an owner. When it ships, the tracker stores the commit and a baseline for that page. It then records checkpoints and only fills the verdict from them. The first sprint under the new tracker moved away from the brand query entirely: a title and description pass for the walker guide, internal links into the electric wheelchair hub, and one supporting article for a category query. Those verdicts are not in yet, and I will update this post with whatever they say, including if they say “flat”.
Key takeaways
- Store baseline, checkpoints and verdict as separate fields. Capture the page’s numbers on ship day, re-measure at fixed intervals, and compute the verdict only from those two. “It shipped” must never be a valid outcome.
- Treat an empty “didn’t work” list as a bug. If your loop has run for a month and found nothing to avoid, check the verdict logic before you celebrate.
- Give the recommender a memory of attempts. Before proposing a change, look up whether the same change type already shipped on the same page recently. Repeat recommendations kill trust in the memo.
- Classify queries before you score them. Separate brand queries you do not own from category queries. Only the second kind should generate title or snippet work.
- Keep the cap at three. The cap was the one part that worked from day one: small enough to approve on a phone, specific enough to ship the same week.
If you want to copy it this week
You do not need an AI agent to start. Export the last 28 days of queries and pages from Search Console into a spreadsheet. Add four columns: change, date shipped, baseline CTR, CTR four weeks later. Pick three rows a week, make the change, and fill the last column honestly a month later. The spreadsheet will teach you more than the automation did in its first two months, because it forces the one step the automation skipped.
The recipe and the memo format live in the open Agentic Company playbook.
— Alex

