pFad - Phone/Frame/Anonymizer/Declutterfier! Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

URL: http://github.com/aboutcode-org/vulnerablecode/pull/2163

Add support for Reference Fix Commits improver by ziadhany · Pull Request #2163 · aboutcode-org/vulnerablecode · GitHub
Skip to content

Add support for Reference Fix Commits improver#2163

Merged
ziadhany merged 10 commits into
aboutcode-org:mainfrom
ziadhany:collect-package-patch
May 14, 2026
Merged

Add support for Reference Fix Commits improver#2163
ziadhany merged 10 commits into
aboutcode-org:mainfrom
ziadhany:collect-package-patch

Conversation

@ziadhany
Copy link
Copy Markdown
Collaborator

@ziadhany ziadhany commented Feb 13, 2026

@ziadhany ziadhany force-pushed the collect-package-patch branch from b9b3fe1 to ae2b35a Compare February 13, 2026 23:21
@ziadhany ziadhany marked this pull request as ready for review February 13, 2026 23:46
@ziadhany ziadhany force-pushed the collect-package-patch branch 2 times, most recently from 8bf03c1 to 374de5e Compare February 20, 2026 02:54
@ziadhany
Copy link
Copy Markdown
Collaborator Author

ziadhany commented Feb 20, 2026 •
edited
Loading

Pipeline Logs:

Improving data using collect_ref_fix_commits_v2
INFO 2026-05-14 21:07:34.249503 UTC Pipeline [CollectReferencesFixCommitsPipeline] starting
INFO 2026-05-14 21:07:34.249611 UTC Step [collect_and_store_fix_commits] starting
INFO 2026-05-14 21:07:51.859808 UTC Progress: 10% (65918/659171) ETA: 152 seconds (2.5 minutes)
INFO 2026-05-14 21:08:10.340882 UTC Progress: 20% (131835/659171) ETA: 142 seconds (2.4 minutes)
INFO 2026-05-14 21:08:46.361722 UTC Progress: 30% (197752/659171) ETA: 167 seconds (2.8 minutes)
INFO 2026-05-14 21:09:57.738196 UTC Progress: 40% (263669/659171) ETA: 214 seconds (3.6 minutes)
INFO 2026-05-14 21:10:30.890334 UTC Progress: 50% (329586/659171) ETA: 176 seconds (2.9 minutes)
INFO 2026-05-14 21:11:02.626930 UTC Progress: 60% (395503/659171) ETA: 138 seconds (2.3 minutes)
INFO 2026-05-14 21:11:33.566542 UTC Progress: 70% (461420/659171) ETA: 102 seconds (1.7 minutes)
INFO 2026-05-14 21:11:56.804078 UTC Progress: 80% (527337/659171) ETA: 65 seconds (1.1 minutes)
INFO 2026-05-14 21:12:14.963733 UTC Progress: 90% (593254/659171) ETA: 31 seconds
INFO 2026-05-14 21:12:32.062363 UTC Progress: 100% (659171/659171)
INFO 2026-05-14 21:12:32.870398 UTC Successfully processed pkg patch commit 89,405
INFO 2026-05-14 21:12:32.870578 UTC Step [collect_and_store_fix_commits] completed in 299 seconds (5.0 minutes)
INFO 2026-05-14 21:12:32.870643 UTC Pipeline completed in 299 seconds (5.0 minutes)

from vulnerabilities.models import PackageCommitPatch
PackageCommitPatch.objects.count()
Out[3]: 37954

@ziadhany ziadhany requested a review from TG1999 February 20, 2026 02:58
ziadhany
ziadhany commented Feb 20, 2026
View reviewed changes
Comment thread vulnerabilities/tests/pipelines/v2_improvers/test_reference_collect_commits_v2.py Outdated
TG1999
TG1999 reviewed Feb 24, 2026
View reviewed changes
Comment thread vulnerabilities/pipelines/v2_improvers/reference_collect_commits.py Outdated
TG1999
TG1999 reviewed Feb 24, 2026
View reviewed changes
Comment thread vulnerabilities/pipelines/v2_improvers/reference_collect_commits.py Outdated
TG1999
TG1999 requested changes Feb 24, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@TG1999 TG1999 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nits for your consideration

Comment thread vulnerabilities/pipelines/v2_improvers/reference_collect_commits.py
Comment thread vulnerabilities/pipelines/v2_improvers/reference_collect_commits.py
@ziadhany ziadhany force-pushed the collect-package-patch branch from db09f78 to bb4550f Compare March 3, 2026 01:32
ziadhany
ziadhany commented Mar 3, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator Author

@ziadhany ziadhany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TG1999 I have updated the pipeline to use bulk, as you suggested.

also, the pipeline can now create an ImpactedPackage if the package does not already have one, so we can associate the PackageCommitPatch with it.

Comment thread vulnerabilities/pipelines/v2_improvers/reference_collect_commits.py
@ziadhany ziadhany requested a review from TG1999 March 3, 2026 01:40
@ziadhany ziadhany mentioned this pull request Apr 14, 2026
Closed
4 tasks
@ziadhany ziadhany mentioned this pull request Apr 30, 2026
Merged
TG1999
TG1999 reviewed May 14, 2026
View reviewed changes
Comment thread vulnerabilities/pipelines/v2_improvers/reference_collect_commits.py Outdated
TG1999
TG1999 reviewed May 14, 2026
View reviewed changes
Comment thread vulnerabilities/pipelines/v2_improvers/reference_collect_commits.py
TG1999
TG1999 reviewed May 14, 2026
View reviewed changes
Comment thread vulnerabilities/pipelines/v2_improvers/reference_collect_commits.py
TG1999
TG1999 reviewed May 14, 2026
View reviewed changes
Comment thread vulnerabilities/pipelines/v2_improvers/reference_collect_commits.py Outdated
TG1999
TG1999 reviewed May 14, 2026
View reviewed changes
Comment thread vulnerabilities/pipelines/v2_improvers/reference_collect_commits.py Outdated
ziadhany added 9 commits May 14, 2026 17:50
@ziadhany
Add support for Reference Fix Commits improver
e53093a 
Update the pipeline and fix the test

Signed-off-by: ziad hany <ziadhany2016@gmail.com>
@ziadhany
Improve the pipeline structure and reduce the number of database queries
2c2c158 
Signed-off-by: ziad hany <ziadhany2016@gmail.com>
@ziadhany
Delete the unnecessary license_expression var.
eda8e81 
Signed-off-by: ziad hany <ziadhany2016@gmail.com>
@ziadhany
Drop unused test
9656668 
Signed-off-by: ziad hany <ziadhany2016@gmail.com>
@ziadhany
Update the pipeline to use bulk
7fe2626 
Signed-off-by: ziad hany <ziadhany2016@gmail.com>
@ziadhany
this pipeline should be registered as improver
acfb3e5 
Signed-off-by: ziad hany <ziadhany2016@gmail.com>
@ziadhany
Try to optimize the CollectReferencesFixCommitsPipeline pipeline
04236f2 
Signed-off-by: ziad hany <ziadhany2016@gmail.com>
@ziadhany
Remove the invalid registration as an importer
fa72a1b 
Signed-off-by: ziad hany <ziadhany2016@gmail.com>
@ziadhany
Update bulk_commit_batch_update to avoid through_model to bulk create…
ddda5cf 
… PackageCommitPatch.fixed_in_impacts

Set ignore_conflicts to True, add id when doing .only(), use iterator

Signed-off-by: ziad hany <ziadhany2016@gmail.com>
@ziadhany ziadhany force-pushed the collect-package-patch branch from df0ce6d to ddda5cf Compare May 14, 2026 14:52
@TG1999
Copy link
Copy Markdown
Contributor

TG1999 commented May 14, 2026

@ziadhany LGTM! add logs in PR and merge and also add completion report. Thanks!

TG1999
TG1999 approved these changes May 14, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@TG1999 TG1999 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@ziadhany ziadhany merged commit 6871324 into aboutcode-org:main May 14, 2026
5 checks passed
@ziadhany ziadhany mentioned this pull request May 14, 2026
Closed
@ziadhany ziadhany deleted the collect-package-patch branch May 15, 2026 10:54
@ziadhany ziadhany mentioned this pull request May 15, 2026
Closed
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@TG1999 TG1999 TG1999 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ziadhany @TG1999
pFad - Phonifier reborn

Pfad - The Proxy pFad © 2024 Your Company Name. All rights reserved.




Check this box to remove all script contents from the fetched content.



Check this box to remove all images from the fetched content.


Check this box to remove all CSS styles from the fetched content.


Check this box to keep images inefficiently compressed and original size.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy