Evaluation-Framework

It is an evaluation framework for evaluating and comparing graph embedding techniques

Entity Relatedness

Datasets used as gold standard

Dataset Structure Size
KORE main entity with a sorted list of 20 related entities 420 entities

Model

sim_scores = []
for each main entity as me:
  for each related entity as re:
    sim_scores.add(similarity_function(me,re))
  sort(sim_scores) //from more to less similar}

The similarity_function can be customized by the user.

Missing entities are managed as follows:

Output of the evaluation

Metric Range Interpretation
Kendall’s tau correlation coefficient [-1,1] Extreme values: correlation, Values close to 0: no correlation