Cover photograph by Yusuf Miah on Pexels.
← Insights & ArticlesFour tools that each did their job, one pipeline that funded a television station.
Aug 23, 2026 · 11 min read
Here is a row from a media plan my pipeline produced. It is the current version, after the fixes. Read it as a person deciding where to send money would read it.
handle one creator
fit_score 61
email comercial@[a television station]
email_source website
contact_note "found on the station's website, which does not obviously
belong to this creator - check before sending"
allocated_usd 43.20
in_plan true
expected_views 2400
relevant_views null
audience_note "audience not measured"It looks careful. There is a warning attached to the address, a note admitting the audience was never measured, and a null where a number would be misleading.
That care is scar tissue. Every one of those columns exists because at some point it did not, and something went wrong in the gap.
The version that mattered had no contact_note. It had an address in a column headed email, a fit score, a dollar figure, and a drafted email ready to send. The address was a television station's advertising desk. In another run it was privacy@linktr.ee, which is Linktree's legal contact and not a person at all.
Nothing on those rows was malformed. Nothing was empty. Every address was syntactically valid and genuinely deliverable. They would pass any email verifier ever written, including the one I wrote. The run was green.
Every check that existed passed, because validity was never the question. The question was whose inbox it is, and no column asked it.
(The pre-fix rows are reconstructed from the decision records written at the time. I have the fixed run on disk and I do not have the broken one, so where this piece describes the earlier state, it is describing it from the record rather than showing it to you.)
Four tools, run in sequence by an n8n workflow: find creators in a niche, measure each channel's audience, read each creator's website to see what they sell, and recover a contact where none was published.
They are four independent programs. None calls another and none knows the others exist. The workflow is the only thing that composes them: it reads one tool's output and constructs the next one's input. That distinction matters for everything below, because every failure I am about to describe happened in a seam, and a seam belongs to whoever wired it, which was me.
Each tool was, as far as I can tell, correct. Each returned data traceable to a real page. The composition produced a plan that allocated a brand's budget to people who would never see it.
This is the part nobody warns you about. There is a great deal written about making a component reliable and almost nothing about what happens when four reliable components are stacked. The failures do not look like bugs. They look like results.
The channel analyzer infers an audience's language from what viewers write in the comments, rather than from whatever the channel declared years ago. When the evidence is too thin to support a call, it refuses to make one: it publishes the distribution it measured and returns audience_language: null.
That refusal has a specific cause. Romanized Hindi defeats language detection at every sample size, so a channel with an Indian audience writing in Latin script reads as English. Rather than say "English" about an audience that is not English, the analyzer says nothing and shows its working. It is the well-behaved component in this story.
One layer up, the template needed something to put in an exclusion column. It wrote audience language undetermined, and then it excluded on that.
Read that again, because it is not what I thought had happened either. The analyzer returned an absence. The consumer invented a word for the absence, and then acted on the word it had invented. Nothing was misread, because there was nothing to read: audience_language was null, relevantViews was null, and two nulls became a verdict with a name.
So the pipeline deleted exactly the channels whose audiences could not be cleanly labelled, which is to say the mixed-language audiences that a multilingual campaign exists to reach. Diaspora markets, border regions and bilingual countries are ordinary commerce, and they were the ones being dropped.
The part that still bothers me: the full language distribution was on the row the whole time. The template read the single-language field sitting next to it. The data needed to make the right call was never missing.
The rule: when a component refuses, go and read what the caller does with the refusal. A well-behaved return value is dangerous precisely because it is well-behaved. Nobody writes a handler for politeness. An exception gets caught and a null gets checked, but an absence that arrives quietly gets a name invented for it by whoever is in a hurry.
The enrichment step reads a creator's website and reports what that business sells. Creators came back selling "Professional Services", "Clean code linters" and "Open journalism".
Those are GitHub's own marketing navigation. The creators had listed their GitHub profiles as their websites, and the tool had read the page it was given, accurately, and reported what that page was selling. Another creator's listed site was an amzn.to affiliate link. Followed to its destination, it reported that he sells "Amazon Haul", "Amazon Secured Card" and "Home Services".
Every fact was read correctly off the page it names. The page was never his.
The rule: a URL a person typed into a profile field is not evidence of what they do. It is evidence of what they linked to. Storefront hosts are the real exception (Etsy, Gumroad, Ko-fi), because there the URL is itself evidence of selling.
The fix was not better reading. It was a hosted_on field: when a page belongs to a platform rather than to the person who listed it, say so. And then, one layer up, the workflow uses that field to decide which sites the contact-finder is allowed to touch at all.
That is worth sitting with. The fix for "a legal contact address ended up in a column headed email" was an edge in the graph, not a validation rule. No amount of checking the address would have caught it, because the address was fine.
The competitor check answers "does this creator already sell something like the thing we are advertising". Part of it reads their website. A separate part reads what brands they name in their own videos.
When a creator's website could not be reached, the check returned early, and took the video-derived signal down with it, despite that signal having no relationship to the website at all.
One unreachable host silently disabled an unrelated detector. Reading the code did not find it. A test written for the case did.
The rule: when a function returns early, enumerate what else was on that path.
When none of the chosen creators had a measurable audience, the headline the run announced was:
$46 reaches an estimated 0 relevant views across 2 creatorsMissing had become zero somewhere in the arithmetic, and zero is a claim. "We could not measure this" and "we measured this and it was nothing" are opposite statements, and only one of them was true.
The same confusion further upstream, treating unmeasured as disqualifying, returned an empty plan twice on briefs that had perfectly good candidates in them.
The rule: unknown is not zero, and the distinction has to survive all the way to whatever a human actually reads. It is not enough to keep null in the store if the sentence at the top renders it as a number.
The headline is now built by a function whose only interesting branch is the one where nothing could be measured:
/**
* The one sentence somebody reads before anything else.
*
* The zero case is the one that matters. When every chosen creator had too thin
* a comment sample, the reach total is 0 - and a live run duly announced
* "$46 reaches an estimated 0 relevant views across 2 creators", which reads as
* a broken campaign rather than an unmeasured one. Missing is not zero, and a
* headline that confuses the two is worse than no headline.
*/
function headlineFor({ chosen, spent, reach, measured, rows }) {
if (!chosen.length) {
return `No creator fits this brief yet. ${whyNot(rows)}`.trim() + chaseLine(rows);
}
const money = `$${Math.round(spent).toLocaleString()}`;
const people = `${chosen.length} creator${chosen.length === 1 ? '' : 's'}`;
const unmeasured = chosen.filter((r) => !measured(r)).length;
if (unmeasured === chosen.length) {
const clause = chosen.length === 1
? 'Their audience could not be measured'
: 'None of their audiences could be measured';
return `${money} across ${people}. ${clause}, so there is no reach estimate `
+ '- missing, not zero.' + chaseLine(rows);
}
return `${money} reaches an estimated ${reach.toLocaleString()} relevant views across ${people}.`
+ (unmeasured ? ` ${unmeasured} of them had too few comments to measure, so `
+ 'their reach is not in that number.' : '')
+ chaseLine(rows);
}And when it chooses nobody, it says why, which is a better artifact than most success paths: "No creator fits this brief yet. 4 with too few comments to read their audience; 2 with no contact we could find; 1 no longer uploading."
Here is the competitor check, which decides whether a creator already sells something like the thing being advertised. It carries the repairs for two of the four failures above.
const MARKET_STOPWORDS = new Set([
'and', 'for', 'the', 'with', 'from', 'your', 'our', 'you', 'all', 'new', 'best',
'shop', 'store', 'buy', 'sale', 'shipped', 'shipping', 'free', 'online', 'made',
'quality', 'premium', 'products', 'product', 'services', 'service', 'company',
'brand', 'official', 'home', 'more', 'get', 'how', 'why', 'top', 'about',
]);
const WORDS = /[^\p{L}\p{N}]+/u;
function marketTerms(brief) {
const raw = [brief.product || '', ...(brief.niches || [])].join(' ');
return [...new Set(raw.toLowerCase().split(WORDS)
.filter((w) => w.length >= 3 && !MARKET_STOPWORDS.has(w)))];
}
function sharedTerms(terms, text) {
const words = new Set(String(text || '').toLowerCase().split(WORDS).filter(Boolean));
return terms.filter((t) => words.has(t)
|| words.has(`${t}s`)
|| (t.endsWith('s') && words.has(t.slice(0, -1))));
}
function ownBrandCheck(profile, analysis, brief) {
const terms = marketTerms(brief);
const readable = profile && profile.status === 'ok';
if (readable && terms.length) {
const catalogue = [profile.sells, profile.brands, profile.product_examples]
.filter(Boolean).join(' | ');
const listed = sharedTerms(terms, catalogue);
if (listed.length) {
return {
verdict: 'yes',
evidence: `lists ${listed.join(', ')} among what they sell`
+ (profile.sells_evidence ? ` - ${profile.sells_evidence}` : ''),
};
}
const described = sharedTerms(terms, profile.description);
if (described.length) {
// "Describes itself" is only true when the page is theirs. A creator whose
// listed website is an Amazon affiliate link is described by Amazon, and
// saying otherwise puts words in his mouth - the row should read the way a
// person would say it out loud.
const whose = profile.hosted_on
? `their ${profile.hosted_on} page mentions`
: 'describes itself using';
return {
verdict: 'possible',
evidence: `${whose} ${described.join(', ')} on ${profile.website}`,
};
}
}
// Checked last but checked *always*. This signal comes from the channel, not
// the website, so a site that timed out - or a creator who has no site at all -
// must not swallow it. It did, until a test asked for the case.
if (Number(analysis.own_product_signals) > 0) {
return {
verdict: 'possible',
evidence: `promotes their own product in ${analysis.own_product_signals} video(s)`,
};
}
return { verdict: null, evidence: null };
}The own-product signal is checked last but always, deliberately outside the branch that needs a readable website. That is incident three's fix: the signal comes from the creator's videos, so a site that timed out, or a creator with no site at all, must not swallow it. The comment in the code says so, because the next person to tidy this will want to move it inside the branch where it looks like it belongs.
The whose variable is incident two's fix in miniature. When a page belongs to a platform, the row reads "their GitHub page mentions" rather than "describes itself using". A creator whose listed website is an affiliate link is described by Amazon, and writing "describes itself" puts words in his mouth. The row should read the way a person would say it out loud.
Note also what the rule does not do. Exclusion requires a catalogue: your market's words appearing in what they actually list for sale, quoted with the page that proves it. Anything weaker is a flag with its evidence attached and a human decides. A single-strength rule firing on "has a shop" would delete every creator selling t-shirts, which is most of the successful ones.
And the price, which is documented rather than fixed: matching is whole words, so "mate" does not match "tomate". The cost is that compounds miss, and a site selling "homelab" hardware does not match a brief that says "home lab". That is reported as a known limit rather than papered over with substring matching, which would manufacture exactly the confident nonsense this layer exists to prevent.
(This section is the reusable part. It stands alone.)
The last step drafts an outreach email with an LLM, from a fact sheet the pipeline assembled. A model writing outreach will, sooner or later, compliment a video that does not exist. It reads beautifully and it is the most embarrassing thing this system could do to a customer: the creator knows instantly that nobody watched anything, and the brand looks worse than if it had sent nothing at all.
So every draft is checked against the facts it was given, before it is saved. Two checks only, chosen because they can be made precisely rather than because they sound thorough:
A draft that fails is still saved, never silently dropped, with needs_review: true and the offending sentence attached. A human decides.
/**
* Check what the model wrote against what it was told.
*
* A model drafting outreach will, sooner or later, compliment a video that does
* not exist. It reads beautifully and it is the single most embarrassing thing
* this template could do to a customer - a creator knows instantly that nobody
* watched anything, and the brand looks worse than if it had sent nothing.
*
* So every draft is checked before it is saved, and the checks are the two that
* can be made precisely rather than the many that sound clever:
*
* 1. **Every number in the draft must have come from the fact sheet.** View
* counts, percentages and budgets are exactly what a model embellishes.
* 2. **No claim to have watched anything.** We never supply a video title, so
* any sentence referring to a specific video is unsupported by construction.
*
* A draft that fails is still saved - never silently dropped - with
* `needs_review: true` and the offending sentence attached. The human decides.
*/
const CONTENT_CLAIM = new RegExp(
'\\b(' +
'i (?:really )?(?:loved|enjoyed|liked|watched|saw|binged)|' +
"i've been (?:watching|following)|" +
'your (?:recent |latest |last )?(?:video|episode|upload|series|short)s? (?:on|about)|' +
'the (?:one|episode|video) (?:where|about|on)' +
')\\b', 'i',
);
function numbersIn(text) {
// 180,000 45500 98% $3,240 1.5k - normalised to bare digits for comparison
const out = [];
const re = /\b\d[\d,.]*\s?k?\b/gi;
let m;
while ((m = re.exec(text)) !== null) {
const raw = m[0].trim();
let value = parseFloat(raw.replace(/,/g, '').replace(/k$/i, ''));
if (/k$/i.test(raw)) value *= 1000;
if (Number.isFinite(value)) out.push({ raw, value });
}
return out;
}
function allowedNumbers(facts) {
const allowed = new Set();
const add = (n) => {
if (n === null || n === undefined || n === '') return;
const v = Number(n);
if (!Number.isFinite(v)) return;
// Floor AND ceil, not just round. An allocation of 121.5 written as "$121"
// is a supplied number rounded, not an invented one - and flagging it made
// the guard cry wolf on a real run, which is worse than missing a catch
// because nobody trusts the next flag.
allowed.add(Math.floor(v));
allowed.add(Math.ceil(v));
allowed.add(Math.round(v));
allowed.add(Math.floor(v / 1000)); // "180,000" written as "180k"
allowed.add(Math.ceil(v / 1000));
allowed.add(Math.floor(v * 100)); // a share written as a percentage
allowed.add(Math.ceil(v * 100));
};
for (const v of Object.values(facts || {})) {
if (typeof v === 'number') add(v);
else if (typeof v === 'string') for (const n of numbersIn(v)) add(n.value);
}
// Small integers are sentence furniture ("a 60-second integration", "Q1").
for (let i = 0; i <= 100; i++) allowed.add(i);
return allowed;
}
function verifyDraft(draft, facts) {
const problems = [];
const allowed = allowedNumbers(facts);
for (const n of numbersIn(draft.body || '')) {
if (!allowed.has(Math.round(n.value))) {
problems.push(`cites a number nobody supplied: "${n.raw}"`);
}
}
const sentences = (draft.body || '').split(/(?<=[.!?])\s+/);
for (const s of sentences) {
if (CONTENT_CLAIM.test(s)) {
problems.push(`claims to have watched something: "${s.trim().slice(0, 120)}"`);
}
}
return {
...draft,
needs_review: problems.length > 0,
review_reasons: problems.join(' | ') || null,
status: problems.length ? 'draft_held_for_review' : 'draft_created',
};
}
// --- n8n adapter -------------------------------------------------------------
const facts = $('Build a fact sheet').all();
return $input.all().map((item, i) => {
const source = facts[i] ? facts[i].json : {};
const checked = verifyDraft({
handle: source.handle,
to: source.to,
subject: 'Sponsorship enquiry - ' + (source.facts ? source.facts.product : ''),
body: item.json.text || item.json.response || item.json.output || '',
}, source.facts || {});
return { json: {
handle: checked.handle,
to: checked.to,
subject: checked.subject,
body: checked.body,
status: checked.status,
needs_review: checked.needs_review,
review_reasons: checked.review_reasons,
allocated_usd: source.allocated_usd,
expected_views: source.expected_views,
} };
});The first live run flagged a draft that was correct.
A creator's allocation was 121.5. The model wrote $121. The allowed set held only Math.round(121.5), which is 122, so 121 was not in it, and the guard reported: cites a number nobody supplied: "121".
Rounding a supplied number is not inventing one. The allowed set now holds the floor, the ceiling and the rounding of every supplied value.
The general version, which is the thing I would want someone to take from this piece if they took nothing else: a guard that fires on a correct draft is worse than one that misses, because nobody trusts the next flag. Precision matters more than recall in anything a human has to triage. One false positive and the human starts skimming; three and the guard is noise wearing a green tick.
Every integer from 0 to 100 is allowed unconditionally. "A 60-second integration", "Q1", "the top 3". All of it sentence furniture. Without that allowance the guard fires constantly and becomes the noise described above. The price is that an invented number under a hundred gets through. That is a deliberate trade, not an oversight.
The adapter pairs the fact sheet to the model output by index. It assumes the chain preserved order. It has so far. It is the first line I would look at if this ever started misbehaving.
The guard has fired twice in its recorded history.
The first was against a mock model we wrote to be wrong on purpose, whose planted errors include a drafted email citing a video title nobody supplied. It caught exactly that. This proves the check functions and proves nothing whatsoever about a real model, because we wrote the sentence in order to catch it.
The second was the 121 false positive above. A real run, a real model, a correct draft, and the guard was wrong.
Every other draft in every real run came back clean.
So: it has never caught a real hallucination. I want to be careful about what that means, because it would be easy to write it as though it said something about how often models invent things. It does not. We have run very few real drafts through it, one or two per campaign across a handful of campaigns. That is far too small a sample to claim anything in either direction. It has caught nothing real yet, and it has not had many chances to.
I am telling you this because a guard whose track record you cannot see is a guard you cannot calibrate. Everything published about LLM guardrails is written by someone with a catch to show. Here is one with no catch, one false positive that was our fault, a deliberate hole you could drive an invented number under a hundred through, and an adapter that assumes ordering. Trust it accordingly, which is roughly how much I trust it.
Not much of the code inside any of the four tools. Almost all of it was at the seams:
- Read what the caller does with every refusal, every null, every undetermined. - Carry provenance in the column beside the value, not in a separate report nobody opens. An address found by crawling is an inference and must be scored below one the person published themselves. - Let one tool's output gate another tool's input, rather than validating harder at the end. - Keep unknown distinct from zero all the way to the sentence a human reads. - Write a test for the case you cannot imagine happening, because the fixtures test the world you already thought of.
That last one is the one that changed how I work. On the newest of these tools the offline suite passed clean; the first live run found defects the fixtures could not have seen; the first live campaign found more again. Fixtures test the world you already imagined. I now budget time to run the thing for real and read the output, not only to build it and test it.
Every incident above was a system doing exactly what it was told, on data that was exactly right, producing an answer that was exactly wrong. There was no stack trace. There was no failed assertion. There was a spreadsheet, and it was full, and it was green.
If you are assembling tools (or agents, which is the same problem with a larger vocabulary), the thing to internalise is that your components being correct is not evidence that your system is. It is not even weak evidence. The failures live in the joins, and nothing in your test suite is looking at the joins.
Built with the creator finder, the channel analyzer, company enrichment and the website email finder. The email validator is deliberately not in that list: the creator finder verifies addresses in-process using the same engine, so calling the validator as well would repeat work already done.