CLIMATE-FEVER*

CLIMATE-FEVER is a dataset adopting the FEVER methodology that consists of 1,535 real-world claims regarding climate-change. Each claim is accompanied by five manually annotated evidence sentences retrieved from Wikipedia that support, refute or do not give enough information to validate the claim. The total dataset thus contains  7,675 claim-evidence pairs. Furthermore, the dataset features challenging claims that relate multiple facets and disputed cases of claims where both supporting and refuting evidence are present.

The dataset is based on a paper  CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims by Thosas Diggelmann, Jordan Boyd-Graber, Jannis Bulian, Massimiliano Ciaramita,and Markus Leippold. The paper was accepted for the Tackling Climate Change with Machine Learning workshop at NeurIPS 2020. The workshop was held online in December 2020.

Download dataset (JSONL, 3 MB)
Access paper at arXiv.org 
Direct download PDF (PDF, 448 KB)

Please cite the following paper for the use of the dataset:

Diggelmann, Thomas; Boyd-Graber, Jordan; Bulian, Jannis; Ciaramita, Massimiliano; Leippold, Markus (2020). CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims. In: Tackling Climate Change with Machine Learning workshop at NeurIPS 2020, Online, 11 December 2020 - 11 December 2020.

The dataset is formatted as a valid JSONL, i.e. each line is a valid JSON document. Each document corresponds to one claim along with five evidence sentences.

• claim_id: unique claim id
• claim: claim text
• claim_label: overall label assigned to claim (based on majority vote on evidences)
• evidences: top five evidence sentences
  • evidence_id: unique evidence id
  • evidence_label: micro-verdict label
  • article: title of source article (Wikipedia page)
  • evidence: evidence sentence
  • entropy: entropy reflecting uncertainty of votes
  • votes: array containing individual votes

{
"claim_id": "0",
"claim": "Global warming is driving polar bears toward extinction",
"claim_label": "SUPPORTS",
"evidences": [
  {
   "evidence_id": "Extinction risk from global warming:170",
   "evidence_label": "NOT_ENOUGH_INFO",
   "article": "Extinction risk from global warming",
   "evidence": "\"Recent Research Shows Human Activity Driving Earth Towards Global Extinction Event\".",
   "entropy": 0.6931471805599453,
   "votes": [
    "SUPPORTS",
    "NOT_ENOUGH_INFO",
    null,
    null,
    null
   ]
  },
  {
   "evidence_id": "Global warming:14",
   "evidence_label": "SUPPORTS",
   "article": "Global warming",
   "evidence": "Environmental impacts include the extinction or relocation of many species as their ecosystems change, most immediately the environments of coral reefs, mountains, and the Arctic.",
   "entropy": 0.0,
   "votes": [
    "SUPPORTS",
    "SUPPORTS",
    null,
    null,
    null
   ]
  },
  {
   "evidence_id": "Global warming:178",
   "evidence_label": "NOT_ENOUGH_INFO",
   "article": "Global warming",
   "evidence": "Rising temperatures push bees to their physiological limits, and could cause the extinction of bee populations.",
   "entropy": 0.6931471805599453,
   "votes": [
    "SUPPORTS",
    "NOT_ENOUGH_INFO",
    null,
    null,
    null
   ]
  },
  {
   "evidence_id": "Habitat destruction:61",
   "evidence_label": "SUPPORTS",
   "article": "Habitat destruction",
   "evidence": "Rising global temperatures, caused by the greenhouse effect, contribute to habitat destruction, endangering various species, such as the polar bear.",
   "entropy": 0.0,
   "votes": [
    "SUPPORTS",
    "SUPPORTS",
    null,
    null,
    null
   ]
  },
  {
   "evidence_id": "Polar bear:1328",
   "evidence_label": "NOT_ENOUGH_INFO",
   "article": "Polar bear",
   "evidence": "\"Bear hunting caught in global warming debate\".",
   "entropy": 0.6931471805599453,
   "votes": [
    "SUPPORTS",
    "NOT_ENOUGH_INFO",
    null,
    null,
    null
   ]
  }
]
}