How the Finnish Bias Tracker scores political bias.
The Finnish Bias Tracker aggregates recent articles from Finland's major news outlets and applies a documented bias-scoring methodology to each one. Sources span the political spectrum and both national languages: from Vasemmistoliitto-affiliated Kansan Uutiset on the left to Perussuomalaiset-affiliated Suomen Uutiset on the right, from the public broadcaster Yle to Swedish-language Hufvudstadsbladet and Svenska Yle.
The goal is not to declare which articles are true, but to make the framing visible. Two outlets covering the same event can describe it in ways that emphasize different facts, choose different sources, and use different language. By placing each article on a -2 to +2 bias scale alongside the LLM's rationale, the project lets readers see those framing differences directly. The scoring methodology, prompts, source classifications, and scoring history are all public — anyone can audit them.
Every scored article receives an integer bias value from −2 to +2. In Finnish political convention, negative values represent the left and positive values the right — the opposite of US conventions where red is the right and blue is the left.
| Score | Indicator | Description |
|---|---|---|
| −2 | -2 | Clear partisan framing, often party-organ content. Loaded language consistent with a particular left-wing political position. |
| −1 | -1 | Mild left lean detectable in framing, source selection, or vocabulary choices. The article still reads as journalism, not advocacy. |
| 0 | 0 | Neutral, balanced, or wire-style reporting. Multiple perspectives represented; descriptive rather than evaluative language. |
| +1 | +1 | Mild right lean detectable in framing, source selection, or vocabulary choices. The article still reads as journalism, not advocacy. |
| +2 | +2 | Clear partisan framing, often party-organ content. Loaded language consistent with a particular right-wing political position. |
The underlying database schema allows scores from −3 to +3, but in practice the current scoring methodology (v1.2) clamps to ±2. Reserve ±3 for editorial cases the model rates as explicit propaganda — extremely rare in the current dataset.
Every article passes through the same scoring pipeline. After scraping, the article's title, body, source, and publication date are sent to an LLM (currently Gemini 2.5 Flash-Lite) with a versioned prompt. The model returns structured JSON containing the bias score, a confidence value, the detected topic, a one-sentence neutral summary, and a list of specific phrases from the article that drove the score.
Each scoring result is stored alongside the article with the prompt version that produced it. When the prompt is revised, the article can be rescored under the new version without losing the historical scoring data. Every article's detail page shows the full scoring rationale and any version history.
The current prompt (v1.2) is below. It is intentionally English-language even though the articles are in Finnish or Swedish — modern LLMs apply the same scoring principles across languages, and the prompt is more readable for the methodology audit if it's in English.
You are an analytical reviewer assessing political bias in Finnish news articles. Your job is to identify *how* an article is framed, not to judge whether its claims are true. You will be given a Finnish (or Swedish-language Finnish) news article. You will return structured JSON evaluating its political bias on a -3 (far left) to +3 (far right) scale, with 0 being center/neutral. CRITICAL PRINCIPLES: 1. Score the article, not the source. A right-leaning outlet can publish a neutral article. A left-leaning outlet can publish a right-leaning piece. Judge the text in front of you. 2. Provide concrete evidence. Every score must be backed by specific examples from the article — loaded words, framing choices, source selection, omissions. 3. Be calibrated. Most news articles are mildly biased or neutral (-1 to +1). Reserve -3 and +3 for explicitly partisan or party-organ content. 4. Distinguish opinion from news. Opinion pieces will naturally be more biased; that's expected. Note article_type accordingly. 5. Confidence should reflect ambiguity. If the article is short, technical, or genuinely balanced, confidence should be lower. BIAS INDICATORS: Left-leaning signals: - Emphasis on inequality, workers' rights, public services, climate action - Sources skew toward unions, NGOs, academics, progressive politicians - Framing of economic policy emphasizes redistribution, social protection - Critical framing of business interests, austerity, immigration enforcement Right-leaning signals: - Emphasis on individual responsibility, market efficiency, traditional values, sovereignty - Sources skew toward business leaders, conservative politicians, security officials - Framing of economic policy emphasizes growth, deregulation, fiscal discipline - Critical framing of welfare programs, immigration, EU integration, climate regulation Neutral/center indicators: - Multiple perspectives represented with similar weight - Descriptive rather than evaluative language - Sourcing across the political spectrum - Wire-style "who/what/when/where" reporting
Three prompt versions have been used so far. Each revision was driven by observed scoring failures on real articles, not by abstract reasoning about what bias detection should look like.
First-pass prompt. Returned bias on a −3 to +3 scale, confidence, topic, and a free-text rationale. Calibration audit found the model consistently scored mainstream articles as 0 (correct) but also scored party-organ articles as 0 or ±1 — under-detecting strong partisan framing. Examples were unstructured prose, hard to audit.
Restructured the prompt to require an array of specific phrases from the article rather than freeform examples. Forced the model to ground its score in concrete textual evidence. Bias detection on partisan sources improved marginally, but the model still avoided the ±2 endpoints even for clear party-organ content.
Added explicit calibration instruction: "Reserve ±3 for explicitly partisan or party-organ content." Recalibrated the indicators for left-leaning and right-leaning signals to be more specific. After v1.2, party-organ sources (Kansan Uutiset, Suomen Uutiset) reliably score ±2 on opinion-heavy articles, and mainstream sources score in the −1 to +1 range as expected.
This project is pre-alpha. The methodology has known weaknesses worth being explicit about.
n=N) alongside averages so readers can
weigh confidence appropriately.Every source currently in the bias tracker, sorted by source-level bias from left to right. Source-level bias is the editorial classification used as a baseline; individual articles can and often do score differently.
| Source | Bias | Language | Articles indexed |
|---|
Every prompt version, every scoring decision, and every source classification is public. The full source code is on GitHub and the project is licensed under AGPL-3.0 — derivative work must remain open-source. The license choice is intentional: a bias-detection methodology that hides its workings is not a methodology, it is an oracle.
Methodology improvements, prompt refinements, and source classification updates are welcome as GitHub issues or pull requests. The project specifically welcomes contributions from people with backgrounds in Finnish media studies, political science, or computational linguistics.