MS MARCO V1 Passage

TREC 2019 TREC 2020 dev

AP		 nDCG@10 R@1K
AP		 nDCG@10 R@1K RR@10 R@1K
Command to generate run on TREC 2019 queries: 
java -cp $fatjar io.anserini.search.SearchCollection \
  -threads 16 \
  -index msmarco-v1-passage \
  -topics dl19-passage \
  -output runs/run.msmarco.bm25.dl19.txt \
  -hits 1000 -bm25
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl19-passage \
  runs/run.msmarco.bm25.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl19-passage \
  runs/run.msmarco.bm25.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl19-passage \
  runs/run.msmarco.bm25.dl19.txt
Command to generate run on TREC 2020 queries: 
java -cp $fatjar io.anserini.search.SearchCollection \
  -threads 16 \
  -index msmarco-v1-passage \
  -topics dl20-passage \
  -output runs/run.msmarco.bm25.dl20.txt \
  -hits 1000 -bm25
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl20-passage \
  runs/run.msmarco.bm25.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl20-passage \
  runs/run.msmarco.bm25.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl20-passage \
  runs/run.msmarco.bm25.dl20.txt
Command to generate run on dev queries: 
java -cp $fatjar io.anserini.search.SearchCollection \
  -threads 16 \
  -index msmarco-v1-passage \
  -topics msmarco-v1-passage.dev \
  -output runs/run.msmarco.bm25.dev.txt \
  -hits 1000 -bm25
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -M 10 -m recip_rank msmarco-passage.dev \
  runs/run.msmarco.bm25.dev.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 msmarco-passage.dev \
  runs/run.msmarco.bm25.dev.txt
Command to generate run on TREC 2019 queries: 
java -cp $fatjar io.anserini.search.SearchCollection \
  -threads 16 \
  -index msmarco-v1-passage.splade-pp-ed \
  -topics dl19-passage.splade-pp-ed \
  -output runs/run.msmarco.splade-pp-ed.cached_q.dl19.txt \
  -impact -pretokenized -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl19-passage \
  runs/run.msmarco.splade-pp-ed.cached_q.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl19-passage \
  runs/run.msmarco.splade-pp-ed.cached_q.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl19-passage \
  runs/run.msmarco.splade-pp-ed.cached_q.dl19.txt
Command to generate run on TREC 2020 queries: 
java -cp $fatjar io.anserini.search.SearchCollection \
  -threads 16 \
  -index msmarco-v1-passage.splade-pp-ed \
  -topics dl20-passage.splade-pp-ed \
  -output runs/run.msmarco.splade-pp-ed.cached_q.dl20.txt \
  -impact -pretokenized -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl20-passage \
  runs/run.msmarco.splade-pp-ed.cached_q.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl20-passage \
  runs/run.msmarco.splade-pp-ed.cached_q.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl20-passage \
  runs/run.msmarco.splade-pp-ed.cached_q.dl20.txt
Command to generate run on dev queries: 
java -cp $fatjar io.anserini.search.SearchCollection \
  -threads 16 \
  -index msmarco-v1-passage.splade-pp-ed \
  -topics msmarco-v1-passage.dev.splade-pp-ed \
  -output runs/run.msmarco.splade-pp-ed.cached_q.dev.txt \
  -impact -pretokenized -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -M 10 -m recip_rank msmarco-passage.dev \
  runs/run.msmarco.splade-pp-ed.cached_q.dev.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 msmarco-passage.dev \
  runs/run.msmarco.splade-pp-ed.cached_q.dev.txt
Command to generate run on TREC 2019 queries: 
java -cp $fatjar io.anserini.search.SearchCollection \
  -threads 16 \
  -index msmarco-v1-passage.splade-pp-ed \
  -topics dl19-passage \
  -output runs/run.msmarco.splade-pp-ed.onnx.dl19.txt \
  -impact -pretokenized -hits 1000 \
  -encoder SpladePlusPlusEnsembleDistil
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl19-passage \
  runs/run.msmarco.splade-pp-ed.onnx.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl19-passage \
  runs/run.msmarco.splade-pp-ed.onnx.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl19-passage \
  runs/run.msmarco.splade-pp-ed.onnx.dl19.txt
Command to generate run on TREC 2020 queries: 
java -cp $fatjar io.anserini.search.SearchCollection \
  -threads 16 \
  -index msmarco-v1-passage.splade-pp-ed \
  -topics dl20-passage \
  -output runs/run.msmarco.splade-pp-ed.onnx.dl20.txt \
  -impact -pretokenized -hits 1000 \
  -encoder SpladePlusPlusEnsembleDistil
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl20-passage \
  runs/run.msmarco.splade-pp-ed.onnx.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl20-passage \
  runs/run.msmarco.splade-pp-ed.onnx.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl20-passage \
  runs/run.msmarco.splade-pp-ed.onnx.dl20.txt
Command to generate run on dev queries: 
java -cp $fatjar io.anserini.search.SearchCollection \
  -threads 16 \
  -index msmarco-v1-passage.splade-pp-ed \
  -topics msmarco-v1-passage.dev \
  -output runs/run.msmarco.splade-pp-ed.onnx.dev.txt \
  -impact -pretokenized -hits 1000 \
  -encoder SpladePlusPlusEnsembleDistil
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -M 10 -m recip_rank msmarco-passage.dev \
  runs/run.msmarco.splade-pp-ed.onnx.dev.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 msmarco-passage.dev \
  runs/run.msmarco.splade-pp-ed.onnx.dev.txt
Command to generate run on TREC 2019 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.cos-dpr-distil \
  -topics dl19-passage.cos-dpr-distil \
  -output runs/run.msmarco.cos-dpr-distil.fp32.cached_q.dl19.txt \
  -efSearch 1000 -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl19-passage \
  runs/run.msmarco.cos-dpr-distil.fp32.cached_q.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl19-passage \
  runs/run.msmarco.cos-dpr-distil.fp32.cached_q.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl19-passage \
  runs/run.msmarco.cos-dpr-distil.fp32.cached_q.dl19.txt
Command to generate run on TREC 2020 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.cos-dpr-distil \
  -topics dl20-passage.cos-dpr-distil \
  -output runs/run.msmarco.cos-dpr-distil.fp32.cached_q.dl20.txt \
  -efSearch 1000 -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl20-passage \
  runs/run.msmarco.cos-dpr-distil.fp32.cached_q.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl20-passage \
  runs/run.msmarco.cos-dpr-distil.fp32.cached_q.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl20-passage \
  runs/run.msmarco.cos-dpr-distil.fp32.cached_q.dl20.txt
Command to generate run on dev queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.cos-dpr-distil \
  -topics msmarco-v1-passage.dev.cos-dpr-distil \
  -output runs/run.msmarco.cos-dpr-distil.fp32.cached_q.dev.txt \
  -efSearch 1000 -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -M 10 -m recip_rank msmarco-passage.dev \
  runs/run.msmarco.cos-dpr-distil.fp32.cached_q.dev.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 msmarco-passage.dev \
  runs/run.msmarco.cos-dpr-distil.fp32.cached_q.dev.txt
Command to generate run on TREC 2019 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.cos-dpr-distil \
  -topics dl19-passage \
  -output runs/run.msmarco.cos-dpr-distil.fp32.onnx.dl19.txt \
  -efSearch 1000 -hits 1000 \
  -encoder CosDprDistil
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl19-passage \
  runs/run.msmarco.cos-dpr-distil.fp32.onnx.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl19-passage \
  runs/run.msmarco.cos-dpr-distil.fp32.onnx.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl19-passage \
  runs/run.msmarco.cos-dpr-distil.fp32.onnx.dl19.txt
Command to generate run on TREC 2020 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.cos-dpr-distil \
  -topics dl20-passage \
  -output runs/run.msmarco.cos-dpr-distil.fp32.onnx.dl20.txt \
  -efSearch 1000 -hits 1000 \
  -encoder CosDprDistil
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl20-passage \
  runs/run.msmarco.cos-dpr-distil.fp32.onnx.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl20-passage \
  runs/run.msmarco.cos-dpr-distil.fp32.onnx.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl20-passage \
  runs/run.msmarco.cos-dpr-distil.fp32.onnx.dl20.txt
Command to generate run on dev queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.cos-dpr-distil \
  -topics msmarco-v1-passage.dev \
  -output runs/run.msmarco.cos-dpr-distil.fp32.onnx.dev.txt \
  -efSearch 1000 -hits 1000 \
  -encoder CosDprDistil
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -M 10 -m recip_rank msmarco-passage.dev \
  runs/run.msmarco.cos-dpr-distil.fp32.onnx.dev.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 msmarco-passage.dev \
  runs/run.msmarco.cos-dpr-distil.fp32.onnx.dev.txt
Command to generate run on TREC 2019 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.cos-dpr-distil.quantized \
  -topics dl19-passage.cos-dpr-distil \
  -output runs/run.msmarco.cos-dpr-distil.int8.cached_q.dl19.txt \
  -efSearch 1000 -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl19-passage \
  runs/run.msmarco.cos-dpr-distil.int8.cached_q.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl19-passage \
  runs/run.msmarco.cos-dpr-distil.int8.cached_q.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl19-passage \
  runs/run.msmarco.cos-dpr-distil.int8.cached_q.dl19.txt
Command to generate run on TREC 2020 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.cos-dpr-distil.quantized \
  -topics dl20-passage.cos-dpr-distil \
  -output runs/run.msmarco.cos-dpr-distil.int8.cached_q.dl20.txt \
  -efSearch 1000 -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl20-passage \
  runs/run.msmarco.cos-dpr-distil.int8.cached_q.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl20-passage \
  runs/run.msmarco.cos-dpr-distil.int8.cached_q.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl20-passage \
  runs/run.msmarco.cos-dpr-distil.int8.cached_q.dl20.txt
Command to generate run on dev queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.cos-dpr-distil.quantized \
  -topics msmarco-v1-passage.dev.cos-dpr-distil \
  -output runs/run.msmarco.cos-dpr-distil.int8.cached_q.dev.txt \
  -efSearch 1000 -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -M 10 -m recip_rank msmarco-passage.dev \
  runs/run.msmarco.cos-dpr-distil.int8.cached_q.dev.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 msmarco-passage.dev \
  runs/run.msmarco.cos-dpr-distil.int8.cached_q.dev.txt
Command to generate run on TREC 2019 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.cos-dpr-distil.quantized \
  -topics dl19-passage \
  -output runs/run.msmarco.cos-dpr-distil.int8.onnx.dl19.txt \
  -efSearch 1000 -hits 1000 \
  -encoder CosDprDistil
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl19-passage \
  runs/run.msmarco.cos-dpr-distil.int8.onnx.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl19-passage \
  runs/run.msmarco.cos-dpr-distil.int8.onnx.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl19-passage \
  runs/run.msmarco.cos-dpr-distil.int8.onnx.dl19.txt
Command to generate run on TREC 2020 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.cos-dpr-distil.quantized \
  -topics dl20-passage \
  -output runs/run.msmarco.cos-dpr-distil.int8.onnx.dl20.txt \
  -efSearch 1000 -hits 1000 \
  -encoder CosDprDistil
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl20-passage \
  runs/run.msmarco.cos-dpr-distil.int8.onnx.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl20-passage \
  runs/run.msmarco.cos-dpr-distil.int8.onnx.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl20-passage \
  runs/run.msmarco.cos-dpr-distil.int8.onnx.dl20.txt
Command to generate run on dev queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.cos-dpr-distil.quantized \
  -topics msmarco-v1-passage.dev \
  -output runs/run.msmarco.cos-dpr-distil.int8.onnx.dev.txt \
  -efSearch 1000 -hits 1000 \
  -encoder CosDprDistil
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -M 10 -m recip_rank msmarco-passage.dev \
  runs/run.msmarco.cos-dpr-distil.int8.onnx.dev.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 msmarco-passage.dev \
  runs/run.msmarco.cos-dpr-distil.int8.onnx.dev.txt
Command to generate run on TREC 2019 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.bge-base-en-v1.5 \
  -topics dl19-passage.bge-base-en-v1.5 \
  -output runs/run.msmarco.bge-base-en-v1.5.fp32.cached_q.dl19.txt \
  -efSearch 1000 -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl19-passage \
  runs/run.msmarco.bge-base-en-v1.5.fp32.cached_q.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl19-passage \
  runs/run.msmarco.bge-base-en-v1.5.fp32.cached_q.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl19-passage \
  runs/run.msmarco.bge-base-en-v1.5.fp32.cached_q.dl19.txt
Command to generate run on TREC 2020 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.bge-base-en-v1.5 \
  -topics dl20-passage.bge-base-en-v1.5 \
  -output runs/run.msmarco.bge-base-en-v1.5.fp32.cached_q.dl20.txt \
  -efSearch 1000 -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl20-passage \
  runs/run.msmarco.bge-base-en-v1.5.fp32.cached_q.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl20-passage \
  runs/run.msmarco.bge-base-en-v1.5.fp32.cached_q.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl20-passage \
  runs/run.msmarco.bge-base-en-v1.5.fp32.cached_q.dl20.txt
Command to generate run on dev queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.bge-base-en-v1.5 \
  -topics msmarco-v1-passage.dev.bge-base-en-v1.5 \
  -output runs/run.msmarco.bge-base-en-v1.5.fp32.cached_q.dev.txt \
  -efSearch 1000 -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -M 10 -m recip_rank msmarco-passage.dev \
  runs/run.msmarco.bge-base-en-v1.5.fp32.cached_q.dev.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 msmarco-passage.dev \
  runs/run.msmarco.bge-base-en-v1.5.fp32.cached_q.dev.txt
Command to generate run on TREC 2019 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.bge-base-en-v1.5 \
  -topics dl19-passage \
  -output runs/run.msmarco.bge-base-en-v1.5.fp32.onnx.dl19.txt \
  -efSearch 1000 -hits 1000 \
  -encoder BgeBaseEn15
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl19-passage \
  runs/run.msmarco.bge-base-en-v1.5.fp32.onnx.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl19-passage \
  runs/run.msmarco.bge-base-en-v1.5.fp32.onnx.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl19-passage \
  runs/run.msmarco.bge-base-en-v1.5.fp32.onnx.dl19.txt
Command to generate run on TREC 2020 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.bge-base-en-v1.5 \
  -topics dl20-passage \
  -output runs/run.msmarco.bge-base-en-v1.5.fp32.onnx.dl20.txt \
  -efSearch 1000 -hits 1000 \
  -encoder BgeBaseEn15
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl20-passage \
  runs/run.msmarco.bge-base-en-v1.5.fp32.onnx.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl20-passage \
  runs/run.msmarco.bge-base-en-v1.5.fp32.onnx.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl20-passage \
  runs/run.msmarco.bge-base-en-v1.5.fp32.onnx.dl20.txt
Command to generate run on dev queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.bge-base-en-v1.5 \
  -topics msmarco-v1-passage.dev \
  -output runs/run.msmarco.bge-base-en-v1.5.fp32.onnx.dev.txt \
  -efSearch 1000 -hits 1000 \
  -encoder BgeBaseEn15
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -M 10 -m recip_rank msmarco-passage.dev \
  runs/run.msmarco.bge-base-en-v1.5.fp32.onnx.dev.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 msmarco-passage.dev \
  runs/run.msmarco.bge-base-en-v1.5.fp32.onnx.dev.txt
Command to generate run on TREC 2019 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.bge-base-en-v1.5.quantized \
  -topics dl19-passage.bge-base-en-v1.5 \
  -output runs/run.msmarco.bge-base-en-v1.5.int8.cached_q.dl19.txt \
  -efSearch 1000 -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl19-passage \
  runs/run.msmarco.bge-base-en-v1.5.int8.cached_q.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl19-passage \
  runs/run.msmarco.bge-base-en-v1.5.int8.cached_q.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl19-passage \
  runs/run.msmarco.bge-base-en-v1.5.int8.cached_q.dl19.txt
Command to generate run on TREC 2020 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.bge-base-en-v1.5.quantized \
  -topics dl20-passage.bge-base-en-v1.5 \
  -output runs/run.msmarco.bge-base-en-v1.5.int8.cached_q.dl20.txt \
  -efSearch 1000 -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl20-passage \
  runs/run.msmarco.bge-base-en-v1.5.int8.cached_q.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl20-passage \
  runs/run.msmarco.bge-base-en-v1.5.int8.cached_q.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl20-passage \
  runs/run.msmarco.bge-base-en-v1.5.int8.cached_q.dl20.txt
Command to generate run on dev queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.bge-base-en-v1.5.quantized \
  -topics msmarco-v1-passage.dev.bge-base-en-v1.5 \
  -output runs/run.msmarco.bge-base-en-v1.5.int8.cached_q.dev.txt \
  -efSearch 1000 -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -M 10 -m recip_rank msmarco-passage.dev \
  runs/run.msmarco.bge-base-en-v1.5.int8.cached_q.dev.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 msmarco-passage.dev \
  runs/run.msmarco.bge-base-en-v1.5.int8.cached_q.dev.txt
Command to generate run on TREC 2019 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.bge-base-en-v1.5.quantized \
  -topics dl19-passage \
  -output runs/run.msmarco.bge-base-en-v1.5.int8.onnx.dl19.txt \
  -efSearch 1000 -hits 1000 \
  -encoder BgeBaseEn15
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl19-passage \
  runs/run.msmarco.bge-base-en-v1.5.int8.onnx.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl19-passage \
  runs/run.msmarco.bge-base-en-v1.5.int8.onnx.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl19-passage \
  runs/run.msmarco.bge-base-en-v1.5.int8.onnx.dl19.txt
Command to generate run on TREC 2020 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.bge-base-en-v1.5.quantized \
  -topics dl20-passage \
  -output runs/run.msmarco.bge-base-en-v1.5.int8.onnx.dl20.txt \
  -efSearch 1000 -hits 1000 \
  -encoder BgeBaseEn15
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl20-passage \
  runs/run.msmarco.bge-base-en-v1.5.int8.onnx.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl20-passage \
  runs/run.msmarco.bge-base-en-v1.5.int8.onnx.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl20-passage \
  runs/run.msmarco.bge-base-en-v1.5.int8.onnx.dl20.txt
Command to generate run on dev queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.bge-base-en-v1.5.quantized \
  -topics msmarco-v1-passage.dev \
  -output runs/run.msmarco.bge-base-en-v1.5.int8.onnx.dev.txt \
  -efSearch 1000 -hits 1000 \
  -encoder BgeBaseEn15
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -M 10 -m recip_rank msmarco-passage.dev \
  runs/run.msmarco.bge-base-en-v1.5.int8.onnx.dev.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 msmarco-passage.dev \
  runs/run.msmarco.bge-base-en-v1.5.int8.onnx.dev.txt
Command to generate run on TREC 2019 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.cohere-embed-english-v3.0 \
  -topics dl19-passage.cohere-embed-english-v3.0 \
  -output runs/run.msmarco.cohere-embed-english-v3.0.fp32.cached_q.dl19.txt \
  -efSearch 1000 -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl19-passage \
  runs/run.msmarco.cohere-embed-english-v3.0.fp32.cached_q.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl19-passage \
  runs/run.msmarco.cohere-embed-english-v3.0.fp32.cached_q.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl19-passage \
  runs/run.msmarco.cohere-embed-english-v3.0.fp32.cached_q.dl19.txt
Command to generate run on TREC 2020 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.cohere-embed-english-v3.0 \
  -topics dl20-passage.cohere-embed-english-v3.0 \
  -output runs/run.msmarco.cohere-embed-english-v3.0.fp32.cached_q.dl20.txt \
  -efSearch 1000 -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl20-passage \
  runs/run.msmarco.cohere-embed-english-v3.0.fp32.cached_q.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl20-passage \
  runs/run.msmarco.cohere-embed-english-v3.0.fp32.cached_q.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl20-passage \
  runs/run.msmarco.cohere-embed-english-v3.0.fp32.cached_q.dl20.txt
Command to generate run on dev queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.cohere-embed-english-v3.0 \
  -topics msmarco-v1-passage.dev.cohere-embed-english-v3.0 \
  -output runs/run.msmarco.cohere-embed-english-v3.0.fp32.cached_q.dev.txt \
  -efSearch 1000 -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -M 10 -m recip_rank msmarco-passage.dev \
  runs/run.msmarco.cohere-embed-english-v3.0.fp32.cached_q.dev.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 msmarco-passage.dev \
  runs/run.msmarco.cohere-embed-english-v3.0.fp32.cached_q.dev.txt
Command to generate run on TREC 2019 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.cohere-embed-english-v3.0.quantized \
  -topics dl19-passage.cohere-embed-english-v3.0 \
  -output runs/run.msmarco.cohere-embed-english-v3.0.int8.cached_q.dl19.txt \
  -efSearch 1000 -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl19-passage \
  runs/run.msmarco.cohere-embed-english-v3.0.int8.cached_q.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl19-passage \
  runs/run.msmarco.cohere-embed-english-v3.0.int8.cached_q.dl19.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl19-passage \
  runs/run.msmarco.cohere-embed-english-v3.0.int8.cached_q.dl19.txt
Command to generate run on TREC 2020 queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.cohere-embed-english-v3.0.quantized \
  -topics dl20-passage.cohere-embed-english-v3.0 \
  -output runs/run.msmarco.cohere-embed-english-v3.0.int8.cached_q.dl20.txt \
  -efSearch 1000 -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m map dl20-passage \
  runs/run.msmarco.cohere-embed-english-v3.0.int8.cached_q.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m ndcg_cut.10 dl20-passage \
  runs/run.msmarco.cohere-embed-english-v3.0.int8.cached_q.dl20.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -l 2 -m recall.1000 dl20-passage \
  runs/run.msmarco.cohere-embed-english-v3.0.int8.cached_q.dl20.txt
Command to generate run on dev queries: 
java -cp $fatjar io.anserini.search.SearchHnswDenseVectors \
  -threads 16 \
  -index msmarco-v1-passage.cohere-embed-english-v3.0.quantized \
  -topics msmarco-v1-passage.dev.cohere-embed-english-v3.0 \
  -output runs/run.msmarco.cohere-embed-english-v3.0.int8.cached_q.dev.txt \
  -efSearch 1000 -hits 1000
Evaluation commands: 
tools/eval/trec_eval.9.0.4/trec_eval -c -M 10 -m recip_rank msmarco-passage.dev \
  runs/run.msmarco.cohere-embed-english-v3.0.int8.cached_q.dev.txt
tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 msmarco-passage.dev \
  runs/run.msmarco.cohere-embed-english-v3.0.int8.cached_q.dev.txt