How we rate every source.
Our methodology is public because trust requires transparency. Here's exactly how bias, factuality, and ownership ratings work.
Bias Ratings
Every news source in TrueFrame receives a bias rating on a 5-point scale. This rating reflects the source's overall editorial perspective as expressed through story selection, framing, headline language, and sourcing patterns.
Left
Consistently frames stories from progressive/liberal perspective. Prioritizes social justice, government intervention, and institutional critique of conservative positions.
Lean Left
Generally frames stories with a moderate liberal perspective. May present both sides but editorial choices favor progressive framing.
Center
Attempts balanced coverage. Presents multiple perspectives without consistent editorial lean. May still have blind spots but doesn't systematically favor left or right framing.
Lean Right
Generally frames stories with a moderate conservative perspective. May present both sides but editorial choices favor conservative framing.
Right
Consistently frames stories from conservative perspective. Prioritizes free market, traditional values, and institutional critique of progressive positions.
Bias ≠ inaccuracy. A source can be strongly Left or Right and still be highly factual. Bias describes the editorial lens, not the truthfulness. The most biased sources in our database include some of the most factually rigorous. Bias and factuality are independent axes.
How we determine bias
Our AI rates sources by analyzing:
- Story selection patterns — what they choose to cover and ignore
- Headline framing — word choice, emphasis, emotional tone
- Source attribution — which experts and officials are quoted
- Contextual framing — what background is included or omitted
- Editorial pattern over time — not any single article
Each source is rated by our AI engine using a structured prompt with explicit criteria. The prompt instructs the model to evaluate based on U.S. political spectrum conventions.
Confidence score
Each bias rating includes a confidence score (0-1). Sources with extensive public track records receive higher confidence. New or niche sources may have lower confidence until more data accumulates.
Factuality Ratings
Very High
Consistently publishes well-sourced, evidence-based reporting. Corrections issued promptly. Strong editorial standards.
High
Generally accurate with occasional minor errors. Sources claims and provides context.
Mixed
Publishes a mix of factual reporting and opinion/editorial that may not be clearly distinguished. Some unsourced claims.
Low
Frequently publishes misleading, out-of-context, or poorly sourced content. May mix news with editorial without clear labeling.
Very Low
Regularly publishes false, fabricated, or deliberately misleading content. Known for conspiracy theories or propaganda.
How we determine factuality
- Source attribution — does the outlet cite primary sources?
- Correction policy — does it issue corrections when wrong?
- Editorial separation — is news clearly separated from opinion?
- Historical accuracy — fact-check record from third-party checkers
- Transparency — does the outlet disclose ownership, funding, methodology?
Who Owns the News
Media ownership affects editorial independence. A source owned by a hedge fund faces different incentive structures than an independent nonprofit. We don't claim ownership determines bias, but we believe you should know who's behind the byline.
Conglomerate
Owned by a large media or non-media corporation (e.g., Disney, News Corp, Warner Bros Discovery).
Private Equity
Owned by a private equity firm or hedge fund. May face pressure for profit optimization that affects editorial resources.
Family / Individual
Owned by an individual or family (e.g., Jeff Bezos / Washington Post, Rupert Murdoch / Fox News).
Government
State-funded or state-controlled media. Editorial independence varies significantly (BBC vs. RT).
Independent / Nonprofit
Independently owned, often nonprofit or reader-funded. May have mission-driven editorial focus.
Other / Unknown
Ownership structure is unclear, complex, or doesn't fit the above categories.
How the AI Works
Source Added
AI Analysis
Agentic AI
Bias Rating
5-point scale
Factuality
5-tier
Ownership
6 categories
Confidence
0-1 score
Rationale
Text explanation
We use our proprietary AI engine with a structured prompt to analyze each source. The engine receives the source's name, known information, and instructions to rate on our defined scales. Each rating includes a text rationale explaining the reasoning.
We chose AI-assisted rating over purely manual rating because:
- We rate 10,000+ sources. Manual rating at this scale is impractical.
- AI provides consistent application of the same criteria across all sources.
- Every rating includes a written rationale that can be audited.
We chose this over pure crowdsourcing because:
- Crowdsourced ratings reflect the crowd's biases, not the source's.
- Consistency degrades as the crowd grows.
- Rationales are more useful than aggregate scores.
How Stories Are Grouped
Article Ingested
Text Similarity Screen
Levenshtein + Jaccard
Embedding Similarity
Cosine similarity
≥ 0.55
Merge into existing story
< 0.55
Create new story
Stories are clustered using a two-pass algorithm. First, articles are compared using headline text similarity (Levenshtein and Jaccard scoring). Articles that pass this initial screen are then compared using embedding cosine similarity (threshold: 0.55). This two-pass approach balances precision with recall: the text similarity pre-filter catches obvious matches quickly, while embedding similarity catches semantically related articles that use different wording.
Once an article clears both checks, it is merged into the closest matching story cluster. Otherwise, it creates a new story.
Once a story has 5 or more articles, we generate an AI summary and per-bias summaries that describe how different sides are covering the event.
What we get wrong
Known Limitations
No rating system is perfect. Here's where ours has known limitations:
AI Ratings Are Not Ground Truth
Our bias and factuality ratings are AI-generated assessments, not independently verified verdicts. They reflect the model's analysis based on its training data and our prompt instructions. Different models or different prompts would produce somewhat different ratings. We publish rationales so you can evaluate whether you agree.
U.S.-Centric Spectrum
Our 5-point bias scale is calibrated to the U.S. political spectrum. Sources from other countries may not map cleanly. A “centrist” outlet in the UK or France may register as “Lean Left” on a U.S.-calibrated scale. We plan to add region-specific calibration in a future update.
Source-Level, Not Article-Level
We rate bias at two levels. Every source has a standing bias rating based on ground truth data from Media Bias/Fact Check, AllSides, and Ad Fontes Media (supplemented by AI research for sources not in these databases). Additionally, individual articles are classified as left, center, or right using AI analysis of their title and description. The story-level bias bar reflects source-level ratings, while per-article bias classification adds a finer-grained layer of analysis.
Clustering Imperfection
Even with a two-pass approach and an embedding threshold of 0.55, clustering can over-merge loosely related stories or under-merge stories with very different headline framing. We continuously monitor and adjust.
Coverage Gaps
We aggregate from 10,000+ sources but that doesn't mean comprehensive coverage. Non-English sources, paywalled content, and very small outlets may be underrepresented.
Your ratings, your feed
We know our AI ratings won't match your assessment for every source. That's by design. On any source's profile page, you can set your own bias rating. Your override affects only your experience:
- Your My News Bias dashboard uses your overrides
- Your Blindspot feed uses your overrides
- Other users are never affected by your changes
We believe this is the right approach: provide a baseline assessment with transparent methodology, then let you customize based on your own informed judgment.
Questions about our methodology?
Social Sentiment Pipeline
Beyond traditional media coverage, TrueFrame captures how the public reacts to major stories in real time. Our Social Sentiment Pipeline aggregates and analyzes posts from decentralized and mainstream social platforms to surface the mood around each story cluster.
Data sources
Bluesky (AT Protocol)
We query the Bluesky public search API using authenticated sessions via the AT Protocol. For each active story cluster, we search for relevant posts by keyword, then match and score them before analysis.
X (Twitter) API
We use the X API v2 search endpoints to query posts matching story keywords and entities. Rate limits are managed via queued polling with exponential backoff.
Matching posts to stories
Social Post
Keyword Search
topics + entities
Match to Story
rank + filter
Sentiment Score
NLP classifier
Social posts from X and Bluesky are matched to stories using keyword-based search. Posts matching a story's key topics and entities are collected, then analyzed for sentiment, political lean, and authenticity.
Sentiment scoring
Matched posts are classified into three sentiment categories using AI-powered analysis:
Bullish
Positive, supportive, or optimistic reaction to the story or its implications.
Neutral
Informational, balanced, or ambivalent. Sharing without strong opinion.
Bearish
Negative, critical, or pessimistic reaction. Opposition or concern.
Aggregation and display
Sentiment scores are aggregated per story and per platform. The overall sentiment gauge displays a weighted breakdown of bullish, neutral, and bearish posts. We weight by engagement (likes + reposts) to surface posts with broader reach, while capping individual post influence to prevent single viral posts from dominating the score.
Each story's sentiment panel shows: the overall sentiment gauge, per-platform breakdowns (Bluesky vs. X), a sample of representative posts from each sentiment category, and aggregate engagement metrics.
Limitations and transparency
Platform Demographics Are Not Representative
Bluesky and X have different user bases with different political leanings and demographics. Sentiment scores reflect those platforms' users, not the general public. We display per-platform breakdowns so you can account for this.
Bot and Astroturf Activity
Social platforms contain bot accounts and coordinated campaigns. We apply basic filtering (account age, engagement ratios) but cannot fully eliminate inauthentic activity. Treat sentiment as directional, not precise.
Sentiment Classification Errors
Sarcasm, irony, and context-dependent language are difficult to classify accurately. Some posts will be misclassified. We continuously evaluate and improve our classification accuracy as we gather more labeled data.
Temporal Bias
Early reactions to a story tend to be more extreme. Sentiment may moderate as more information becomes available. The gauge reflects the current aggregate and does not yet show sentiment over time (planned for a future update).