helm
Stanford CRFMgithub.com/stanford-crfm/helm
Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducible and transparent evaluation of foundation models, including large language models (LLMs) and multi
Updated 1 h ago · first seen 11 Sept 2026
framework_01M294NQV1YNXYG1KN3DHW0RXD
- Version
- 0.5.16
- T2 · 1 h ago
- Released
- 30 Apr 2026
- T2 · 1 h ago
- License
- Apache-2.0
- T2 · 1 h ago
- Stars
- 2,907
- T2 · 1 h ago
Specification
- License
- Apache-2.0
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 1 h agomedium
- Repository
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 1 h agomedium
- Created at
- 29 Nov 2021
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 1 h agomedium
- Default branch
- main
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 1 h agomedium
- Homepage
- https://crfm.stanford.edu/helm
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 1 h agomedium
- Kind
- tool
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 1 h agomedium
- Latest release
- 30 Apr 2026
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 1 h agomedium
- Latest release tag
- v0.5.16
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 1 h agomedium
- Latest version
- 0.5.16
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 1 h agomedium
- License name
- Apache License 2.0
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 1 h agomedium
- Forks
- 413
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 1 h agomedium
- Releases
- 25
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 1 h agomedium
- Stars
- 2,907
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 1 h agomedium
- Tags
- 25
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 1 h agomedium
- Watchers
- 39
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 1 h agomedium
- PyPI
- crfm-helm
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 1 h agomedium
- Readme url
- https://raw.githubusercontent.com/stanford-crfm/helm/HEAD/README.md
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 1 h agomedium
- Releases url
- https://github.com/stanford-crfm/helm/releases
Source:GitHub public repositories (HTML, releases.atom, raw files)T2observed 1 h agomedium
Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →
Provenance
Attributed facts
19
Source tiers
T219
Freshest observation
1 h ago
Conflicts
None
- Developed by
- Stanford CRFM
As of
Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.
Claim history
Licenselicense1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| Apache-2.0 | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Repositoryrepository_url1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://github.com/stanford-crfm/helm | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Created atcreated_at1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 29 Nov 2021 | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Default branchdefault_branch1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| main | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Descriptiondescription1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducible and transparent evaluation of foundation models, including large language models (LLMs) and multimodal models. | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Homepagehomepage1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://crfm.stanford.edu/helm | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Kindkind1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| tool | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Latest releaselatest_release_at1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 30 Apr 2026 | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Latest release taglatest_release_tag1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| v0.5.16 | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Latest versionlatest_version1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 0.5.16 | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
License namelicense_name1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| Apache License 2.0 | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Forksmetric.forks1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 413 | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Releasesmetric.releases1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 25 | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Starsmetric.stars1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 2,907 | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Watchersmetric.watchers1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 39 | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
PyPIpypi1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| crfm-helm | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Readme urlreadme_url1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://raw.githubusercontent.com/stanford-crfm/helm/HEAD/README.md | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Releases urlreleases_url1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://github.com/stanford-crfm/helm/releases | → current | current | GitHub public repositories (HTML, releases.atom, raw files)T2 | medium | deterministic |
Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →
| Source | Document | Type | Tier | Last observed | Snapshots |
|---|---|---|---|---|---|
| GitHub public repositories (HTML, releases.atom, raw files) | github.com/stanford-crfm/helm/releases.atom | feed | T2· Quality secondary | 1 h ago | 1 |
| GitHub public repositories (HTML, releases.atom, raw files) | raw.githubusercontent.com/stanford-crfm/helm/HEAD/README.md | readme | T2· Quality secondary | 1 h ago | 1 |
| GitHub public repositories (HTML, releases.atom, raw files) | github.com/stanford-crfm/helm | repository | T2· Quality secondary | 1 h ago | 2 |
Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.