evals
Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
Updated 12 min ago · first seen 11 Sept 2026
framework_01M294NHSHM7YQDAFJSC10DRCG
- Version
- 2.0.0
- T2 · 17 min ago
- Released
- 12 Jan 2024
- T2 · 17 min ago
- Stars
- 19,436
- T2 · 12 min ago
Specification
- Repository
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 17 min agomedium
- Created at
- 23 Jan 2023
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 17 min agomedium
- Default branch
- main
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 17 min agomedium
- Kind
- tool
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 17 min agomedium
- Latest release
- 12 Jan 2024
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 17 min agomedium
- Latest release tag
- 2.0.0
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 17 min agomedium
- Latest version
- 2.0.0
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 17 min agomedium
- License name
- Other
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 17 min agomedium
- Forks
- 3,083
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 12 min agomedium
- Stars
- 19,436
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 12 min agomedium
- Watchers
- 282
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 12 min agomedium
- Readme url
- https://raw.githubusercontent.com/openai/evals/HEAD/README.md
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 17 min agomedium
- Releases url
- https://github.com/openai/evals/releases
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 17 min agomedium
Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →
Provenance
Attributed facts
14
Source tiers
T214
Freshest observation
12 min ago
Conflicts
None
- Developed by
- OpenAI
As of
Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.
Claim history
Repositoryrepository_url1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://github.com/openai/evals | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Created atcreated_at1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 23 Jan 2023 | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Default branchdefault_branch1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| main | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Descriptiondescription1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks. | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Kindkind1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| tool | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Latest releaselatest_release_at1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 12 Jan 2024 | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Latest release taglatest_release_tag1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 2.0.0 | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Latest versionlatest_version1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 2.0.0 | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
License namelicense_name1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| Other | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Forksmetric.forks1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 3,083 | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Starsmetric.stars1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 19,436 | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Watchersmetric.watchers1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 282 | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Readme urlreadme_url1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://raw.githubusercontent.com/openai/evals/HEAD/README.md | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Releases urlreleases_url1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://github.com/openai/evals/releases | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →
| Source | Document | Type | Tier | Last observed | Snapshots |
|---|---|---|---|---|---|
| GitHub public repositories (HTML, releases.atom, raw files) | github.com/openai/evals/releases.atom | feed | T2· Quality secondary | 12 min ago | 1 |
| GitHub public repositories (HTML, releases.atom, raw files) | raw.githubusercontent.com/openai/evals/HEAD/README.md | readme | T2· Quality secondary | 12 min ago | 1 |
| GitHub public repositories (HTML, releases.atom, raw files) | github.com/openai/evals | repository | T2· Quality secondary | 12 min ago | 2 |
Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.