iMINDBench

2. Describe your model

Clone this website repository and open its leaderboard/ directory. Choose a model ID such as "example_model" and use that ID in every path and command below.

Create data/submissions/example_model.json:

{
  "schema_version": 1,
  "model_id": "example_model",
  "display_name": "Example Model",
  "pretrained": true,
  "pretrained_on": "Public Dataset",
  "description": "Short public description of the evaluated model.",
  "coverage_note": null
}

Use null for pretrained_on if the model is not pretrained. Leave coverage_note as null unless the coverage section below applies.

3. Generate the artifact

From the leaderboard/ directory, generate the leaderboard artifact from your iMINDBench results:

python add_model.py \
  --model-outputs /path/to/evaluation_outputs/example_model \
  --submission data/submissions/example_model.json \
  --dry-run

The --dry-run option outputs a summary without writing anything. Check the summary, especially the coverage and cohort lines. If it looks right, run the same command again without --dry-run. That creates data/models/example_model.json and updates data/manifest.json and data_bundle.js.

4. Check, then open the pull request

From the leaderboard/ directory, run the validator:

python validate_data.py

Commit only these four files:

• data/submissions/example_model.json
• data/models/example_model.json
• data/manifest.json
• data_bundle.js

Don't commit raw outputs or changes to another model. The pull request template asks for your submission file, the coverage lines from step 3, and details about the model. The same validator runs on your pull request.

You can view the website locally to make sure it appears as intended.

Coverage

Evaluate on the whole benchmark if you can. If you cannot, cover exactly the Main cohort or exactly the Challenge cohort, and use the same choice throughout. Any other subset is rejected; step 3 tells you which one you have.

A cohort submission is listed with an automatic footnote about the subject-sessions it skipped. Enter your cohort name in coverage_note, and the name will be added to that footnote.

For more information about viewing the site locally or updating a previously added model, see leaderboard/README.md.