Skip to content

docs: add new benchmark page with SIFT1M dataset#691

Merged
JoanFM merged 82 commits into
mainfrom
chore-benchmark-refactor-298
Nov 10, 2022
Merged

docs: add new benchmark page with SIFT1M dataset#691
JoanFM merged 82 commits into
mainfrom
chore-benchmark-refactor-298

Conversation

@AnneYang720

@AnneYang720 AnneYang720 commented Oct 25, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor

Goals:

This PR will add sift1m dataset to benchmark scripts according to #298.
The code will continue from the work of #514 , #301 and #310.

Change PR #614 to jina branch.

  • adapt benchmark script to latest docarray
  • add sift1m dataset
  • reuse code in both random and dataset benchmarking
  • add code to generate benchmark recall lines
  • update benchmarking results

alaeddine-13 and others added 30 commits October 12, 2022 15:43
@alaeddine-13 @AnneYang720
fix: adapt benchmark script to latest docarray
c1c32d0
@alaeddine-13 @AnneYang720
chore: increase index size
77b3070
@alaeddine-13 @AnneYang720
chore: add exclude backends
8bebacd
@alaeddine-13 @AnneYang720
fix: ignore errors in drop
6fa645d
@alaeddine-13 @AnneYang720
chore: ef_runtime for redis
3f1ea34
@alaeddine-13 @AnneYang720
fix: fix exclude
4c8f2cc
@alaeddine-13 @AnneYang720
chore: remove print
0f47ee3
fix: update data type of columns
859c6da
fix: fix benchmark_df append
51110c1
@AnneYang720
feat: add benchmark adapted for sift1m
904c620
@davidbp @AnneYang720
fix: order
39ed230
@davidbp @AnneYang720
fix: drop sqlite in table
ae5fa9c
@davidbp @AnneYang720
fix: delete docarray for once finished
c5d7f36
@alaeddine-13 @AnneYang720
fix: correctly drop sqlite
250e5ad
@davidbp @AnneYang720
feat: speedup benchmark from groundtruth
1aa8259
@davidbp @AnneYang720
refactor: sift1M
cabb2f4
@davidbp @AnneYang720
fix: recall bug
09518ff
@davidbp @AnneYang720
refactor: get groundtruth before
8f6c375
@AnneYang720
fix: backend config and filters
e294afb
@AnneYang720
fix: ignore drop error
b3259b1
@AnneYang720
refactor: adjust funcs to be same as random
dc27a64
@AnneYang720
fix: fix elasticsearch connection timeout
c96273c
@AnneYang720
refactor: remove duplicated code
0b2c814
@AnneYang720
refactor: reuse code
08bbee3
@AnneYang720
refactor: reuse code in both random and sift1m
50ddac9
@AnneYang720
fix: fix metric
6cf9d29
@AnneYang720
feat: add hnsw params benchmark
7b819b5
@AnneYang720
feat: add sift benchmark recall/query lines
4cfc38c
@AnneYang720
fix: remove test subset
e4c8b1d
@AnneYang720
refactor: change hnsw params
74499dd
@github-actions

github-actions Bot commented Nov 9, 2022

Copy link
Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

@github-actions

github-actions Bot commented Nov 9, 2022

Copy link
Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

@github-actions

github-actions Bot commented Nov 9, 2022

Copy link
Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

@github-actions

github-actions Bot commented Nov 9, 2022

Copy link
Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

alaeddine-13
alaeddine-13 requested changes Nov 9, 2022
View reviewed changes

@alaeddine-13 alaeddine-13 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's wait until weaviate PR is merged and use the same batch_size for all: 64
We also need to mention this fact in benchmark.md

alaeddine-13
alaeddine-13 reviewed Nov 9, 2022
View reviewed changes
Comment thread docs/advanced/document-store/benchmark.md Outdated
@alexcg1 alexcg1 removed their assignment Nov 9, 2022
alaeddine-13
alaeddine-13 previously requested changes Nov 9, 2022
View reviewed changes
Comment thread docs/advanced/document-store/benchmark.md Outdated
Comment thread docs/advanced/document-store/benchmark.md
@github-actions

github-actions Bot commented Nov 9, 2022

Copy link
Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

@AnneYang720
docs: update benchamrk results and words
e175834 
Signed-off-by: AnneY <evangeline-lun@foxmail.com>
@github-actions

github-actions Bot commented Nov 10, 2022

Copy link
Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

alaeddine-13
alaeddine-13 reviewed Nov 10, 2022
View reviewed changes
Comment thread docs/advanced/document-store/benchmark.md Outdated
@github-actions

github-actions Bot commented Nov 10, 2022

Copy link
Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

1 similar comment
@github-actions

github-actions Bot commented Nov 10, 2022

Copy link
Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

@github-actions

github-actions Bot commented Nov 10, 2022

Copy link
Copy Markdown

📝 Docs are deployed on https://ft-chore-benchmark-refactor-298--jina-docs.netlify.app 🎉

@JoanFM JoanFM changed the title feat: add sift1m to benchmark docs: add new benchmark page with SIFT1M dataset Nov 10, 2022
@JoanFM JoanFM merged commit 938c061 into main Nov 10, 2022
@JoanFM JoanFM deleted the chore-benchmark-refactor-298 branch November 10, 2022 09:54
@alexcg1 alexcg1 mentioned this pull request Nov 14, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alaeddine-13 alaeddine-13 alaeddine-13 left review comments

@JoanFM JoanFM JoanFM approved these changes

+2 more reviewers

@alexcg1 alexcg1 alexcg1 requested changes

@NicholasDunham NicholasDunham NicholasDunham approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

area/docs area/setup size/xl

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@AnneYang720 @bwanglzu @JoanFM @alaeddine-13 @codecov-commenter @alexcg1 @NicholasDunham @davidbp