TREC 2019 | TREC 2020 | dev | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
AP |
nDCG@10 | R@1K | AP |
nDCG@10 | R@1K | RR@10 | R@1K | ||||||
BM25 (k1=0.9, b=0.4) | 0.3013 | 0.5058 | 0.7501 | 0.2856 | 0.4796 | 0.7863 | 0.1840 | 0.8526 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
SPLADE++ EnsembleDistil (cached queries) | 0.5053 | 0.7317 | 0.8726 | 0.5001 | 0.7198 | 0.8995 | 0.3830 | 0.9831 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
SPLADE++ EnsembleDistil (ONNX) | 0.5050 | 0.7308 | 0.8728 | 0.4999 | 0.7197 | 0.8998 | 0.3828 | 0.9831 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
cosDPR-distil w/ HNSW fp32 (cached queries) | 0.4660 | 0.7250 | 0.8222 | 0.4876 | 0.7025 | 0.8540 | 0.3887 | 0.9764 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
cosDPR-distil w/ HNSW fp32 (ONNX) | 0.4660 | 0.7250 | 0.8222 | 0.4876 | 0.7025 | 0.8540 | 0.3887 | 0.9765 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
cosDPR-distil w/ HNSW int8 (cached queries) | 0.4662 | 0.7240 | 0.8219 | 0.4872 | 0.7004 | 0.8538 | 0.3897 | 0.9764 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
cosDPR-distil w/ HNSW int8 (ONNX) | 0.4664 | 0.7247 | 0.8218 | 0.4871 | 0.6996 | 0.8538 | 0.3899 | 0.9764 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
bge-base-en-v1.5 w/ HNSW fp32 (cached queries) | 0.4436 | 0.7065 | 0.8481 | 0.4648 | 0.6780 | 0.8483 | 0.3574 | 0.9788 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
bge-base-en-v1.5 w/ HNSW fp32 (ONNX) | 0.4486 | 0.7016 | 0.8441 | 0.4626 | 0.6768 | 0.8526 | 0.3575 | 0.9788 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
bge-base-en-v1.5 w/ HNSW int8 (cached queries) | 0.4400 | 0.7016 | 0.8481 | 0.4623 | 0.6738 | 0.8420 | 0.3572 | 0.9774 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
bge-base-en-v1.5 w/ HNSW int8 (ONNX) | 0.4454 | 0.7017 | 0.8436 | 0.4596 | 0.6767 | 0.8468 | 0.3575 | 0.9772 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
cohere-embed-english-v3.0 w/ HNSW fp32 (cached queries) | 0.4883 | 0.6956 | 0.8612 | 0.5068 | 0.7245 | 0.8682 | 0.3647 | 0.9751 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
cohere-embed-english-v3.0 w/ HNSW int8 (cached queries) | 0.4888 | 0.6955 | 0.8643 | 0.5052 | 0.7262 | 0.8616 | 0.3656 | 0.9752 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|