Pre-launch NeurIPS challenge draft

BioKG-Align: Link Prediction and Typing over Ontology-Derived Biomedical Knowledge Graphs

BioKG-Align reframes biomedical ontology matching as relation-aware candidate ranking over knowledge graphs. Systems receive a source entity and a fixed set of target candidates, then rank candidate-relation pairs for equivalence or directional subsumption.

How to participate Review data format

NCIT

DOID

SNOMED

FMA

OMIM

ORDO

candidate relation ranking

Status Participant kit available; official dataset release in preparation.

Platform CodaBench Competition bundle in preparation.

Data host TBD Candidate: Hugging Face Dataset with Zenodo DOI snapshot.

Overview

Biomedical alignment for graph, text, and neuro-symbolic methods

Biomedical knowledge is distributed across independently developed ontologies and controlled vocabularies. Aligning these resources supports search, data integration, clinical research, machine learning, and auditable knowledge base construction.

The challenge exposes ontology alignment through a familiar machine learning interface: typed link prediction over fixed candidate sets. Participants may use graph triples, node properties, and optional Datalog rules derived from ontology structure.

Task

Rank candidate-relation pairs across ontology pairs

Input

Each query provides a source entity, a target ontology, and a fixed list of target candidates. The current scaffold uses exactly 30 candidates per query.

Output

Submissions score candidate-relation pairs using the official relation names: equivalent, source_subsumed_by_target, and source_subsumes_target.

Correctness

A prediction is relevant only when both the target entity and the relation type match the hidden reference alignment.

Planned ontology-pair tasks
Task	Source	Target	Current role
`NCIT-DOID`	NCIT	DOID	Disease and cancer concept alignment
`OMIM-ORDO`	OMIM	ORDO	Rare disease and inherited disorder alignment
`SNOMED-FMA`	SNOMED CT	FMA	Clinical anatomy to anatomy alignment
`SNOMED-NCIT`	SNOMED CT	NCIT	Clinical concept to cancer and disease alignment

Data

Public graph package plus hidden-label evaluation

Pre-launch note: final ontology versions, licenses, file counts, train/validation/test sizes, and download URLs are TBD. This public kit contains a tiny example dataset, scorer, validator, baselines, and documentation; it does not generate the official dataset.

Expected public release files
Path	Purpose	Status
`graph/triples.csv`	Typed intra-ontology graph triples and released training anchors.	Format documented; official release TBD.
`graph/properties.csv`	Node identifiers, ontology membership, labels, synonyms, definitions, and permitted metadata.	Format documented; license-cleared release TBD.
`graph/rules.dl`	Selected Datalog-compatible rules derived from ontology axioms.	Format documented; final rule projection TBD.
`alignments/train.tsv`, `alignments/valid.tsv`	Public supervised labels with relation types.	Format documented; final splits TBD.
`tasks//.cands.tsv`	Candidate files for train, validation, and test queries.	Format documented; official candidate sets TBD.
`evaluation/*`	Sample submission, schema, and local scorer interface.	Kit command available; CodaBench integration in preparation.

Public by design

Participants receive graph files, public train and validation labels, public candidate sets, baseline code, documentation, manifests, and a local validation scorer.

Private by design

Hidden test answers, full hidden references, split construction details, licensed raw sources, and official hidden scorer assets are not part of this public kit.

Evaluation

Relation-aware ranking with typed predictions

The primary leaderboard metric is planned to be macro-averaged relation-aware nDCG@10 across ontology-pair tasks. A ranked item is relevant only if the target entity and relation type are both correct.

Secondary metrics from the current scaffold include MRR, Hits@1, Hits@5, Hits@10, MAP, and macro-F1 over relation types. Confidence intervals and significance testing are TBD.

Primary relation-aware nDCG@10

Ranking MRR, Hits@K, MAP

Typing macro-F1 over relation types

Final checks code and environment reproducibility TBD

Baselines

Starting points for several participant communities

Lexical

Ranks candidates using preferred labels, synonyms, and string similarity.

Hybrid lexical

Combines lexical similarity with simple relation priors from the current scaffold.

Graph embedding

TBD Planned baseline over graph triples and training anchors.

Text-enhanced

TBD Planned baseline using labels, synonyms, and definitions.

Rule-aware

TBD Planned baseline demonstrating use of Datalog-derived features.

Protocol

Participation flow

Register Competition platform CodaBench.
Download data Public dataset URL TBD; candidate host is under review.
Train and validate Use public graph files, training labels, validation labels, and the starting kit.
Submit predictions Upload scored candidate-relation pairs in the official TSV format.
Finalize Top teams will provide code, environment details, and reproduction instructions.

Expected submission columns
Column	Description
`SrcEntity`	Source entity identifier from the query.
`TgtEntity`	One target entity from the provided candidate list.
`Relation`	One of the official relation names.
`Score`	Numeric confidence score used for ranking.

Rules

Draft contest rules

Participants must submit predictions in the official format.
Predictions must score only candidate entities provided in the evaluation files.
Released triples, node properties, Datalog rules, training anchors, and validation labels may be used.
External resources and pretrained models may be used if they are public, cited, and declared.
Hidden test labels, unpublished reference alignments, and manual annotation of the hidden test set are not allowed.
Final leaderboard submissions may require code and sufficient reproduction instructions.
Submitted systems are research prototypes and must not claim clinical validity.

Final rules, external-resource policy, team limits, submission limits, and reproduction requirements are TBD.

Schedule

Preparation timeline

Tentative schedule placeholders
Phase	Target	Readiness
Dataset and license finalization	TBD	Ontology versions, redistribution plan, and source permissions under review.
Starting kit release	TBD	Fixture pipeline and scorer scaffold exist; real baselines pending.
Beta test	TBD	Planned with ontology matching, KG learning, and neuro-symbolic reviewers.
Competition launch	TBD	Depends on NeurIPS decision, data readiness, and platform setup.
Final submission and reproducibility checks	TBD	Policy and compute resources to be finalized.

FAQ

Pre-launch questions

Is the final real dataset available?

Not yet. This repository contains a small runnable example and participant tools. Final ontology versions, counts, and download links are TBD.

Will participants need OWL expertise?

No. The challenge interface is graph-learning-ready: triples, node properties, Datalog rules, and candidate TSV files. Ontology-specific background will be documented for participants who want it.

Can external biomedical resources be used?

The draft plan allows public, cited, and declared external resources. The final policy is TBD and will be published before launch.

How is hidden ground truth protected?

Hidden test labels are not included in public candidate files, graph triples, released alignments, or Datalog rules. The official scorer will keep hidden answers server-side.

Organizers

Team and contact

The organizing team will combine expertise in ontology matching, biomedical ontologies, knowledge graph alignment, machine learning, neuro-symbolic reasoning, software engineering, and benchmark evaluation.

Organizer names, affiliations, dedicated contact email, sponsors, awards, and platform administrators are TBD.

Dedicated email TBD

Repository TBD This repository, once public.

Issue tracker or forum TBD