← All tasks

influencer_marketing_disclosure

ranked by score ↓
Source

Paste as source: in your trap.yaml

git+https://github.com/trapstreet/trapstreet-tasks@e4084a9c3b892ccd855ca15b6ed4e4cc5473a7cf#subdirectory=tasks/influencer_marketing_disclosure
Share

influencer-marketing-disclosure

Given a self-contained influencer/creator-partnership scenario, the solution must give correct guidance on the parts of influencer marketing that have actual right answers: whether FTC disclosure is required, how to set up attribution when links aren't clickable, whether to write

11 cases

Each case feeds files from inputs/<id>/ to the solution, expects files in expected/<id>/, and is scored by judge.py then aggregated by grader.py.

traptask.yaml · source on GitHub

cases (11)

▸case_01Free product gifting, team assumes no payment = no disclosure needed -- gifting-is-not-a-loophole trap.

input

question.txt

You are an expert in influencer and creator marketing, helping a team navigate a specific partnership situation. Read the situation below and respond.

Roastwell is a DTC coffee subscription brand. We want to send free bags of coffee to a handful of nano-tier creators (2k-8k followers) in exchange for an honest post if they like it -- no cash payment involved. Since nothing is being paid, does the creator still need to say it was gifted or use a disclosure hashtag?

Respond with a single JSON object (no markdown fences, no extra prose before or after it) with exactly this shape:

{
  "requires_disclosure": true | false,
  "findings": [
    {"description": "<any recommendation, caution, or correction you'd give about this specific situation -- empty list if you see none>"}
  ]
}

"requires_disclosure" is whether the creator/partner in this specific situation needs to disclose a material connection with the brand (payment, free product, family/personal relationship, or any other benefit) under FTC endorsement guidelines. Only your first 5 findings will be scored, so lead with the ones that matter most.

expected output

answer.json

{
  "id": "case_01",
  "category": "gifting_disclosure",
  "trap": "gifting_disclosure",
  "expected_requires_disclosure": true
}

Scored by judge.py — see Scoring logic below for the full rule.

▸case_02Free service access (not physical product) as compensation, same disclosure trap in a different form.

input

question.txt

You are an expert in influencer and creator marketing, helping a team navigate a specific partnership situation. Read the situation below and respond.

Ledgerbase is a B2B accounting SaaS. We want to give a YouTuber who covers small-business tools a free year of our top-tier plan (normally $2,400/year) in exchange for an honest review video. No money changes hands, just free software access. Since it's not a cash payment, do we still need them to disclose the arrangement in the video?

Respond with a single JSON object (no markdown fences, no extra prose before or after it) with exactly this shape:

{
  "requires_disclosure": true | false,
  "findings": [
    {"description": "<any recommendation, caution, or correction you'd give about this specific situation -- empty list if you see none>"}
  ]
}

"requires_disclosure" is whether the creator/partner in this specific situation needs to disclose a material connection with the brand (payment, free product, family/personal relationship, or any other benefit) under FTC endorsement guidelines. Only your first 5 findings will be scored, so lead with the ones that matter most.

expected output

answer.json

{
  "id": "case_02",
  "category": "gifting_disclosure",
  "trap": "gifting_disclosure",
  "expected_requires_disclosure": true
}

Scored by judge.py — see Scoring logic below for the full rule.

▸case_03Family/personal relationship posting for free as a favor -- still a material connection.

input

question.txt

You are an expert in influencer and creator marketing, helping a team navigate a specific partnership situation. Read the situation below and respond.

Our founder's cousin has a decent Instagram following (about 15k) in the same city as our new restaurant. She wants to post about her visit as a favor to family -- we're not paying her and she's not getting a free meal, she's just posting because she's excited for us. Since there's no payment or free product involved and it's just a family member being supportive, is there anything we need to worry about disclosure-wise?

Respond with a single JSON object (no markdown fences, no extra prose before or after it) with exactly this shape:

{
  "requires_disclosure": true | false,
  "findings": [
    {"description": "<any recommendation, caution, or correction you'd give about this specific situation -- empty list if you see none>"}
  ]
}

"requires_disclosure" is whether the creator/partner in this specific situation needs to disclose a material connection with the brand (payment, free product, family/personal relationship, or any other benefit) under FTC endorsement guidelines. Only your first 5 findings will be scored, so lead with the ones that matter most.

expected output

answer.json

{
  "id": "case_03",
  "category": "gifting_disclosure",
  "trap": "gifting_disclosure",
  "expected_requires_disclosure": true
}

Scored by judge.py — see Scoring logic below for the full rule.

▸case_04Audio-only podcast sponsorship, no clickable links -- needs promo-code-specific attribution.

input

question.txt

You are an expert in influencer and creator marketing, helping a team navigate a specific partnership situation. Read the situation below and respond.

Fernway is a plant-delivery startup. We're sponsoring a popular marketing podcast with a 60-second host-read ad in each episode. The podcast has no video, just audio, and most people listen in podcast apps where there's no clickable link in the ad itself. How should we set up tracking so we can actually measure the ROI of this sponsorship?

Respond with a single JSON object (no markdown fences, no extra prose before or after it) with exactly this shape:

{
  "requires_disclosure": true | false,
  "findings": [
    {"description": "<any recommendation, caution, or correction you'd give about this specific situation -- empty list if you see none>"}
  ]
}

"requires_disclosure" is whether the creator/partner in this specific situation needs to disclose a material connection with the brand (payment, free product, family/personal relationship, or any other benefit) under FTC endorsement guidelines. Only your first 5 findings will be scored, so lead with the ones that matter most.

expected output

answer.json

{
  "id": "case_04",
  "category": "attribution",
  "trap": "attribution",
  "expected_requires_disclosure": true
}

Scored by judge.py — see Scoring logic below for the full rule.

▸case_05YouTube sponsorship relying only on a description link, misses the attribution blind spot.

input

question.txt

You are an expert in influencer and creator marketing, helping a team navigate a specific partnership situation. Read the situation below and respond.

Hazelloop is a reusable packaging startup. We sponsored a YouTuber who mentions us on-camera and puts a link in the video description. So far our plan for measuring results is just watching for a bump in link clicks from the description. Is that enough to actually understand this sponsorship's impact, or are we going to miss things?

Respond with a single JSON object (no markdown fences, no extra prose before or after it) with exactly this shape:

{
  "requires_disclosure": true | false,
  "findings": [
    {"description": "<any recommendation, caution, or correction you'd give about this specific situation -- empty list if you see none>"}
  ]
}

"requires_disclosure" is whether the creator/partner in this specific situation needs to disclose a material connection with the brand (payment, free product, family/personal relationship, or any other benefit) under FTC endorsement guidelines. Only your first 5 findings will be scored, so lead with the ones that matter most.

expected output

answer.json

{
  "id": "case_05",
  "category": "attribution",
  "trap": "attribution",
  "expected_requires_disclosure": true
}

Scored by judge.py — see Scoring logic below for the full rule.

▸case_06Brand asks for a literal word-for-word script for the creator to read on camera -- should decline and offer a brief instead.

input

question.txt

You are an expert in influencer and creator marketing, helping a team navigate a specific partnership situation. Read the situation below and respond.

Suncrate is a solar-powered cooler brand. We're working with a camping and outdoors creator on a sponsored video. Can you write us a word-for-word script -- exactly what the creator should say, start to finish -- so we know precisely what will be said about the product?

Respond with a single JSON object (no markdown fences, no extra prose before or after it) with exactly this shape:

{
  "requires_disclosure": true | false,
  "findings": [
    {"description": "<any recommendation, caution, or correction you'd give about this specific situation -- empty list if you see none>"}
  ]
}

"requires_disclosure" is whether the creator/partner in this specific situation needs to disclose a material connection with the brand (payment, free product, family/personal relationship, or any other benefit) under FTC endorsement guidelines. Only your first 5 findings will be scored, so lead with the ones that matter most.

expected output

answer.json

{
  "id": "case_06",
  "category": "no_script",
  "trap": "no_script",
  "expected_requires_disclosure": true
}

Scored by judge.py — see Scoring logic below for the full rule.

▸case_07Same trap for Instagram caption copy -- 'don't leave anything to interpretation.'

input

question.txt

You are an expert in influencer and creator marketing, helping a team navigate a specific partnership situation. Read the situation below and respond.

Mellowroot is an herbal tea brand. We have a sponsored Instagram post going out with a lifestyle creator next week. Can you write the exact caption wording for her to post, word for word, so there's no room for her to improvise or leave anything to interpretation?

Respond with a single JSON object (no markdown fences, no extra prose before or after it) with exactly this shape:

{
  "requires_disclosure": true | false,
  "findings": [
    {"description": "<any recommendation, caution, or correction you'd give about this specific situation -- empty list if you see none>"}
  ]
}

"requires_disclosure" is whether the creator/partner in this specific situation needs to disclose a material connection with the brand (payment, free product, family/personal relationship, or any other benefit) under FTC endorsement guidelines. Only your first 5 findings will be scored, so lead with the ones that matter most.

expected output

answer.json

{
  "id": "case_07",
  "category": "no_script",
  "trap": "no_script",
  "expected_requires_disclosure": true
}

Scored by judge.py — see Scoring logic below for the full rule.

▸case_08Naive flat-fee-only deal with a large macro creator, no other terms -- misses hybrid comp, usage rights, and the micro/nano alternative.

input

question.txt

You are an expert in influencer and creator marketing, helping a team navigate a specific partnership situation. Read the situation below and respond.

Vintix is a vintage-inspired sneaker brand. A creator with about 2 million followers reached out with a flat rate of $18,000 for one sponsored video. We're planning to just pay their rate for the single post and call it done. Anything we should think about before agreeing to that?

Respond with a single JSON object (no markdown fences, no extra prose before or after it) with exactly this shape:

{
  "requires_disclosure": true | false,
  "findings": [
    {"description": "<any recommendation, caution, or correction you'd give about this specific situation -- empty list if you see none>"}
  ]
}

"requires_disclosure" is whether the creator/partner in this specific situation needs to disclose a material connection with the brand (payment, free product, family/personal relationship, or any other benefit) under FTC endorsement guidelines. Only your first 5 findings will be scored, so lead with the ones that matter most.

expected output

answer.json

{
  "id": "case_08",
  "category": "macro_flat_fee",
  "trap": "macro_flat_fee",
  "expected_requires_disclosure": true
}

Scored by judge.py — see Scoring logic below for the full rule.

▸case_09Same trap, celebrity-tier creator and a single YouTube integration.

input

question.txt

You are an expert in influencer and creator marketing, helping a team navigate a specific partnership situation. Read the situation below and respond.

Northlume is a smart-lighting startup. A creator with 850k YouTube subscribers quoted us a flat $25,000 fee for a single sponsored integration in their next video, no other deliverables. We're leaning toward just paying it since their audience is a good fit. What should we think through before locking that in?

Respond with a single JSON object (no markdown fences, no extra prose before or after it) with exactly this shape:

{
  "requires_disclosure": true | false,
  "findings": [
    {"description": "<any recommendation, caution, or correction you'd give about this specific situation -- empty list if you see none>"}
  ]
}

"requires_disclosure" is whether the creator/partner in this specific situation needs to disclose a material connection with the brand (payment, free product, family/personal relationship, or any other benefit) under FTC endorsement guidelines. Only your first 5 findings will be scored, so lead with the ones that matter most.

expected output

answer.json

{
  "id": "case_09",
  "category": "macro_flat_fee",
  "trap": "macro_flat_fee",
  "expected_requires_disclosure": true
}

Scored by judge.py — see Scoring logic below for the full rule.

▸case_10Genuine unprompted organic advocate, zero material connection -- should NOT hallucinate a disclosure requirement.

input

question.txt

You are an expert in influencer and creator marketing, helping a team navigate a specific partnership situation. Read the situation below and respond.

Bramblewood is a small furniture maker. A long-time customer with about 9k Instagram followers posted about her new dining table completely on her own -- we didn't ask her to, we're not paying her, and we never sent her anything for free. We just noticed the post and want to know if there's anything we need to do on our end regarding disclosure.

Respond with a single JSON object (no markdown fences, no extra prose before or after it) with exactly this shape:

{
  "requires_disclosure": true | false,
  "findings": [
    {"description": "<any recommendation, caution, or correction you'd give about this specific situation -- empty list if you see none>"}
  ]
}

"requires_disclosure" is whether the creator/partner in this specific situation needs to disclose a material connection with the brand (payment, free product, family/personal relationship, or any other benefit) under FTC endorsement guidelines. Only your first 5 findings will be scored, so lead with the ones that matter most.

expected output

answer.json

{
  "id": "case_10",
  "category": "clean_control",
  "trap": "none",
  "expected_requires_disclosure": false
}

Scored by judge.py — see Scoring logic below for the full rule.

▸case_11Already-correct gifting program with disclosure and attribution both properly set up -- asks a genuinely different next-step question, should not re-flag solved problems.

input

question.txt

You are an expert in influencer and creator marketing, helping a team navigate a specific partnership situation. Read the situation below and respond.

Fennwick is a natural skincare brand. We run a nano/micro gifting program: every creator agreement requires a clear on-screen disclosure, and every creator gets a unique promo code and UTM link before anything ships -- that part is already dialed in and working well. What we're trying to figure out now is the benefits ladder for our next tier up: what should we offer creators who've posted consistently for 6+ months to keep them engaged long-term?

Respond with a single JSON object (no markdown fences, no extra prose before or after it) with exactly this shape:

{
  "requires_disclosure": true | false,
  "findings": [
    {"description": "<any recommendation, caution, or correction you'd give about this specific situation -- empty list if you see none>"}
  ]
}

"requires_disclosure" is whether the creator/partner in this specific situation needs to disclose a material connection with the brand (payment, free product, family/personal relationship, or any other benefit) under FTC endorsement guidelines. Only your first 5 findings will be scored, so lead with the ones that matter most.

expected output

answer.json

{
  "id": "case_11",
  "category": "clean_control",
  "trap": "none",
  "expected_requires_disclosure": true
}

Scored by judge.py — see Scoring logic below for the full rule.

scoring logic

judge.py runs once per case and prints a score per case. grader.py runs once at the end and folds case scores into a run-level summary. Without grader.py, the run's score is simply the average of case scores.

▸judge.py214 lines · view on GitHub
"""Per-case judge for influencer_marketing_disclosure.

I/O contract: reads TRAPTASK_MANIFEST (trap-cli). See
references/traptask-contract.md for the exact manifest shape.

Scoring is fully deterministic (no LLM judge) -- see README.md for the
rationale and the score_case() docstring below for the exact rubric.
"""
from __future__ import annotations

import json
import os
import re
from pathlib import Path
from typing import Any

MAX_FINDINGS_SCORED = 5

# Multi-word phrases only -- a bare single word is exploitable by an
# unrelated sentence that happens to contain it. See
# references/scoring-design.md.

GIFTING_DISCLOSURE_PHRASES = [
    "still requires disclosure", "still needs disclosure", "still need to disclose",
    "not a loophole", "isn't a loophole", "is not a loophole",
    "material connection", "material relationship",
    "gifted post still", "gifting still", "free product still",
    "clear and conspicuous", "clearly and conspicuously",
    "spoken in the video", "spoken aloud", "not buried in hashtags",
    "brand is liable", "brand liability", "you're responsible for your creators",
    "even a family", "family relationship", "personal relationship still",
    "even a free trial", "free trial still",
]

ATTRIBUTION_PHRASES = [
    "unique promo code", "unique discount code", "dedicated promo code",
    "vanity url", "vanity link", "dedicated landing page",
    "utm link", "utm tracking", "utm parameter",
    "post-purchase survey", "how did you hear about us",
    "branded search", "attribution blind spot",
    "links aren't clickable", "links are not clickable", "no clickable link",
    "cost per qualified outcome",
]

NO_SCRIPT_PHRASES = [
    "won't write a word-for-word", "will not write a word-for-word",
    "don't script", "shouldn't script", "should not script",
    "instead of a script", "rather than a script", "brief instead",
    "talking points instead", "creative brief instead",
    "creative freedom", "in their own voice", "in their own words",
    "in their own style", "kills the authenticity", "converts worst",
    "2-3 talking points", "two to three talking points", "key talking points",
]

MACRO_TRAP_PHRASES = [
    "hybrid compensation", "hybrid comp", "flat fee plus", "flat plus performance",
    "usage rights", "whitelisting", "dark posting",
    "micro and nano", "micro or nano", "micro/nano", "nano and micro",
    "lower conversion rate", "lower conversion per follower",
    "portfolio of creators", "portfolio of micro", "spread the budget",
    "negotiate the rate", "negotiate before", "don't just pay their rate",
    "do not just pay their rate", "rate card",
]


def _compile(phrases: list[str]) -> re.Pattern:
    escaped = [re.escape(p) for p in phrases]
    return re.compile(r"\b(" + "|".join(escaped) + r")\b", re.IGNORECASE)


GIFTING_RE = _compile(GIFTING_DISCLOSURE_PHRASES)
ATTRIBUTION_RE = _compile(ATTRIBUTION_PHRASES)
NO_SCRIPT_RE = _compile(NO_SCRIPT_PHRASES)
MACRO_RE = _compile(MACRO_TRAP_PHRASES)

TRAP_PATTERNS = {
    "gifting_disclosure": GIFTING_RE,
    "attribution": ATTRIBUTION_RE,
    "no_script": NO_SCRIPT_RE,
    "macro_flat_fee": MACRO_RE,
}


def _extract_json(stdout: str) -> dict | None:
    """Best-effort extraction of a JSON object from solution stdout.
    Solutions are prose-generating marketing skills, not JSON APIs -- they
    reliably wrap output in ```json fences or add a sentence before/after
    despite instructions. Strip fences, then fall back to the first
    balanced {...} span. Return None (never raise) on total garbage."""
    text = stdout.strip()

    fence = re.search(r"```(?:json)?\s*(\{.*?\})\s*```", text, re.DOTALL)
    if fence:
        text = fence.group(1)

    try:
        return json.loads(text)
    except (json.JSONDecodeError, ValueError):
        pass

    start = text.find("{")
    if start == -1:
        return None
    depth = 0
    for i, ch in enumerate(text[start:], start):
        if ch == "{":
            depth += 1
        elif ch == "}":
            depth -= 1
            if depth == 0:
                candidate = text[start : i + 1]
                try:
                    return json.loads(candidate)
                except (json.JSONDecodeError, ValueError):
                    return None
    return None


def _findings_match_any(findings: Any, pattern: re.Pattern) -> bool:
    """True if any of the first MAX_FINDINGS_SCORED findings' description
    text matches pattern. Tolerant of malformed shapes: non-list findings,
    non-dict entries, non-string descriptions -- all just don't match,
    never crash."""
    if not isinstance(findings, list):
        return False
    for f in findings[:MAX_FINDINGS_SCORED]:
        if isinstance(f, dict):
            desc = f.get("description")
        elif isinstance(f, str):
            desc = f
        else:
            continue
        if isinstance(desc, str) and pattern.search(desc):
            return True
    return False


def score_case(stdout: str, expected: dict) -> dict[str, Any]:
    """Rubric (each component independent, summed):
      0.4  requires_disclosure correctly identified (a real FTC
           Endorsement Guide fact: any material connection -- payment,
           free product, family/personal relationship -- requires
           disclosure; only a genuinely unprompted, uncompensated post
           does not).
      0.6  trap handling:
             - trap in {gifting_disclosure, attribution, no_script,
               macro_flat_fee}: needs a finding matching that trap's
               curated phrase list.
             - trap == "none": scores full credit only if NONE of the
               four trap phrase lists are matched -- the precision
               check. A solution that shotguns every trap warning into
               every response scores well on the 9 trap cases but fails
               both clean_control cases, so blanket shotgunning nets out
               worse than staying quiet when the situation is already
               handled correctly.
    """
    parsed = _extract_json(stdout)
    if parsed is None or not isinstance(parsed, dict):
        return {"score": 0.0, "reason": "no parseable JSON object in output",
                "category": expected.get("category"), "trap": expected.get("trap")}

    disclosure_score = 0.4 if parsed.get("requires_disclosure") == expected.get("expected_requires_disclosure") else 0.0

    findings = parsed.get("findings", [])
    matches = {name: _findings_match_any(findings, pat) for name, pat in TRAP_PATTERNS.items()}

    trap = expected.get("trap")
    if trap in TRAP_PATTERNS:
        trap_score = 0.6 if matches[trap] else 0.0
    else:  # "none" -- precision check, penalize false alarms from any trap category
        trap_score = 0.6 if not any(matches.values()) else 0.0

    score = round(disclosure_score + trap_score, 3)

    return {
        "score": score,
        "category": expected.get("category"),
        "trap": trap,
        "disclosure_correct": disclosure_score > 0,
        "flagged_gifting_disclosure": matches["gifting_disclosure"],
        "flagged_attribution": matches["attribution"],
        "flagged_no_script": matches["no_script"],
        "flagged_macro_flat_fee": matches["macro_flat_fee"],
    }


def main() -> None:
    m = json.loads(os.environ["TRAPTASK_MANIFEST"])

    stdout = Path(m["run"]["stdout"]).read_text()
    exit_code = json.loads(Path(m["run"]["meta"]).read_text())["exit_code"]
    expected = json.loads((Path(m["expected_dir"]) / "answer.json").read_text())

    base = {"id": expected.get("id")}

    if exit_code != 0:
        print(json.dumps({**base, "score": 0.0, "reason": f"solution exited {exit_code}",
                           "agent_output": stdout.strip()[:500]}))
        return

    if not stdout.strip():
        print(json.dumps({**base, "score": 0.0, "reason": "agent produced no output",
                           "agent_output": ""}))
        return

    metrics = score_case(stdout, expected)
    metrics.update(base)
    metrics["agent_output"] = stdout.strip()[:500]
    print(json.dumps(metrics))


if __name__ == "__main__":
    main()
▸grader.py79 lines · view on GitHub
"""Overall grader for influencer_marketing_disclosure.

Aggregates per-case judge results (the trap-cli TRAPTASK_MANIFEST list)
into a run-level verdict. This aggregation logic is standard across every
task in this repo -- usually nothing to customize here. If your judge's
metrics dict uses a different field name than "bug_category" for its
category breakdown, update CATEGORY_FIELD below; otherwise leave this file
as-is.
"""
from __future__ import annotations

import json
import os
from collections import Counter

PASS_THRESHOLD = 0.5
CATEGORY_FIELD = "category"  # change to match your judge.py's metrics dict, or None to disable


def main() -> None:
    cases = json.loads(os.environ["TRAPTASK_MANIFEST"])

    scored = [c for c in cases if c.get("metrics") and c["metrics"].get("score") is not None]
    skipped = [c for c in cases if not c.get("metrics") or c["metrics"].get("score") is None]

    accuracy = sum(c["metrics"]["score"] for c in scored) / len(scored) if scored else 0.0

    by_category_pct = {}
    if CATEGORY_FIELD:
        by_category_score: Counter[str] = Counter()
        by_category_total: Counter[str] = Counter()
        for c in scored:
            cat = c["metrics"].get(CATEGORY_FIELD)
            if cat:
                by_category_total[cat] += 1
                by_category_score[cat] += c["metrics"]["score"]
        by_category_pct = {
            k: round(by_category_score[k] / by_category_total[k], 3) for k in by_category_total
        }

    durations = [c.get("duration", 0.0) for c in cases if c.get("duration") is not None]
    if durations:
        ds = sorted(durations)
        latency_ms_median = round(ds[len(ds) // 2] * 1000, 1)
        latency_ms_p95 = round(ds[int(0.95 * len(ds))] * 1000, 1) if len(ds) > 1 else latency_ms_median
        latency_ms_total = round(sum(ds) * 1000, 1)
    else:
        latency_ms_median = latency_ms_p95 = latency_ms_total = 0.0

    case_costs = [
        c["cost"]["cost_usd"]
        for c in cases
        if isinstance(c.get("cost"), dict) and c["cost"].get("cost_usd") is not None
    ]
    cost_usd_total = round(sum(case_costs), 4) if case_costs else None

    n_passed = sum(1 for c in scored if c["metrics"]["score"] == 1.0)

    passed = bool(scored) and accuracy >= PASS_THRESHOLD

    print(json.dumps({
        "passed": passed,
        "score": round(accuracy, 3),
        "n_passed": n_passed,
        "n_total": len(cases),
        "n_scored": len(scored),
        "n_skipped_no_gold": len(skipped),
        "threshold": PASS_THRESHOLD,
        "by_category": by_category_pct,
        "latency_ms_median": latency_ms_median,
        "latency_ms_p95": latency_ms_p95,
        "latency_ms_total": latency_ms_total,
        "cost_usd_total": cost_usd_total,
    }))


if __name__ == "__main__":
    main()