Content Grader

Compare human, AI, and hybrid content. Score it. Keep the winners.

● All data stays in your browser

1Set the brief

What was the original task? This gives the grader (and future you) the context to judge fairly.

Experiment title

Content type optional

The brief / original prompt

Audience / context optional

2Paste the three versions

Leave any version blank if you only want to compare two.

Human Author optional

Content

AI Source optional

Content

Hybrid Author + tool optional

Content

3Get AI grading

Generate a structured prompt, run it through Claude (or another LLM), then paste the JSON response back. The tool fills in proposed scores you can override.

requires API key in Settings

Or use the manual workflow:

Copied to clipboard

Paste the AI's JSON response here

Tips for fair grading

Score the writing, not the source. Mask which version is which when you can — bias creeps in fast.
Always override the AI's scores if they feel off. AI judges are useful starting points, not arbiters.
Check accuracy claims yourself for anything factual — AI judges struggle with verification.
Run the same brief multiple times before drawing conclusions. One sample is anecdote.

4Review & override scores

Each criterion is scored 1–5: tone fit, factual accuracy, freedom from unfair bias, and how well the content adheres to the brief. Edit anything the AI got wrong.

Notes any context you want to remember

5Save & export

Saved

JSONL is one line per record — the format most fine-tuning pipelines expect. Saved experiments live in your browser; export them periodically to a folder you back up.

Rubric weights

Multiply each criterion's score by its weight before summing. Default 1×. Range 0–2. Use this to make accuracy count more than tone, or to drop bias entirely (set to 0).

Saved

AI grading optional

Skip the copy/paste workflow and grade in one click. Pick the mode that fits how you're using the tool.

Use this when you (or someone) has deployed the proxy/ Cloudflare Worker. The API key stays server-side; users only need the proxy URL and shared password if one is set. See proxy/README.md for deploy steps.

Proxy URL

Shared password if the proxy requires one

Model

Data

Everything is stored in this browser. Export periodically — clearing browser data wipes saved experiments.

How this works

This tool runs a small experiment: you give it three versions of the same content (human, AI, hybrid), grade them on tone, accuracy, and bias, and keep the highest-scoring version. Over time you build a library of winning examples you can use to train or evaluate language models.

The grading workflow

Brief: Describe the task each version was responding to. Without context, "good" is impossible to judge.
Versions: Paste up to three versions. Author/source labels are optional but useful for later analysis.
AI grading: Click Generate grading prompt. The tool builds a structured prompt with all three versions and the rubric, and copies it to your clipboard. Paste it into Claude (or any LLM that returns JSON), and paste the JSON response back into the tool.
Override: AI judges are a starting point. Edit any score you disagree with — your judgement is the ground truth.
Save: The experiment is stored in your browser. Export winners as JSONL when you're ready to use them for training.

The rubric

Each criterion is scored 1 (poor) to 5 (excellent):

Tone — How well the voice, register, and style fit the intended audience and purpose.
Accuracy — Factual correctness, internal consistency, absence of fabricated claims.
Bias — Balance and freedom from unjustified slant. Higher score = less biased.
Brief fit — How well the content adheres to the brief's structural and format requirements (topics covered, section order, word count, format constraints). Score 3 if no brief was provided.

Total is the sum (max 20). Winner = highest total. Ties are broken in order: accuracy, brief fit, tone.

Data & privacy

Everything is stored in your browser's localStorage for this file. Nothing is sent anywhere. If you clear your browser data — or open the file in a different browser — your saved experiments are gone, so export a JSON backup periodically.

JSONL export format

Each line is one record, structured for training pipelines:

{"prompt": "...", "completion": "...", "source": "human", "scores": {"tone": 5, "accuracy": 4, "bias": 5, "briefFit": 4, "total": 18}, "rationale": {...}, "metadata": {"title": "...", "createdAt": "...", "experimentId": "..."}}

Risk notes

AI judges are imperfect. They can be confidently wrong, especially on accuracy and bias. Always sanity-check.
Small samples mislead. One winning AI output doesn't mean AI is "better." Build a corpus before drawing conclusions.
Selection bias. If you only train on winners, you encode your rubric — including its blind spots — into the model.
Local storage is fragile. Browsers can wipe it. Export backups.

1Set the brief

2Paste the three versions

3Get AI grading

4Review & override scores

5Save & export

Saved experiments

Rubric weights

AI grading optional

Data

How this works

The grading workflow

The rubric

Data & privacy

JSONL export format

Risk notes

Preview