Content-Length: 466746 | pFad | https://github.com/python/cpython/pull/128736

B7 gh-118761: substitute `re` import in `base64.b16decode` for a more efficient alternative by picnixz · Pull Request #128736 · python/cpython · GitHub
Skip to content

gh-118761: substitute re import in base64.b16decode for a more efficient alternative#128736

Merged
AA-Turner merged 8 commits intopython:mainfrom
picnixz:perf/import/base64-118761
Jan 14, 2025
Merged

gh-118761: substitute re import in base64.b16decode for a more efficient alternative#128736
AA-Turner merged 8 commits intopython:mainfrom
picnixz:perf/import/base64-118761

Conversation

@picnixz
Copy link
Copy Markdown
Member

@picnixz picnixz commented Jan 11, 2025 •
edited
Loading

Benchmarks are on a RELEASE build (no PGO, no LTO).

See #128736 (comment) for the runtime performance improvements as well.

PR

$ ./python -I -X importtime -c 'import base64'
import time: self [us] | cumulative | imported package
...
import time:       179 |        179 | linecache
import time:       205 |        205 |     _struct
import time:       442 |        646 |   struct
import time:       227 |        227 |   binascii
import time:       298 |       1169 | base64
$ hyperfine --warmup 16 "./python -c 'import base64'"
Benchmark 1: ./python -c 'import base64'
  Time (mean ± σ):       5.7 ms ±   0.5 ms    [User: 4.8 ms, System: 1.1 ms]
  Range (min … max):     5.2 ms …  11.9 ms    455 runs

Main

$ ./python -I -X importtime -c 'import base64'
import time: self [us] | cumulative | imported package
...
import time:       180 |        180 | linecache
import time:       312 |        312 |       types
import time:      1609 |       1921 |     enum
import time:        87 |         87 |       _sre
import time:       250 |        250 |         re._constants
import time:       388 |        637 |       re._parser
import time:        89 |         89 |       re._casefix
import time:       339 |       1150 |     re._compiler
import time:       103 |        103 |         itertools
import time:        83 |         83 |         keyword
import time:        58 |         58 |           _operator
import time:       206 |        264 |         operator
import time:       129 |        129 |         reprlib
import time:        49 |         49 |         _collections
import time:       785 |       1411 |       collections
import time:        43 |         43 |       _functools
import time:       543 |       1997 |     functools
import time:       122 |        122 |     copyreg
import time:       826 |       6015 |   re
import time:       115 |        115 |     _struct
import time:        74 |        189 |   struct
import time:       122 |        122 |   binascii
import time:       279 |       6603 | base64
$ hyperfine --warmup 16 "./python -c 'import base64'"
Benchmark 1: ./python -c 'import base64'
  Time (mean ± σ):       9.5 ms ±   0.4 ms    [User: 8.2 ms, System: 1.3 ms]
  Range (min … max):     9.1 ms …  13.5 ms    295 runs

@picnixz
Improve import time of the base64 module.
3a92fce 
Importing `base64` is now up to six times faster.

The `re` module is now locally imported by `base64.b16decode`
and is no more implicitly exposed as `base64.re`.
@picnixz
blurb
0b52712
@bedevere-app bedevere-app bot mentioned this pull request Jan 11, 2025
Closed
@picnixz picnixz added the performance Performance or resource usage label Jan 11, 2025
@picnixz picnixz requested a review from hugovk January 11, 2025 16:30
@picnixz
Copy link
Copy Markdown
Member Author

picnixz commented Jan 11, 2025

@hugovk I'm requesting your review since you've commented on the issue. There are also #128732 and #128738 to review FYI.

AA-Turner
AA-Turner reviewed Jan 12, 2025
View reviewed changes
@picnixz
Improve base64.b16decode() by a factor 10.
d7b6253 
This entirely removes the needs to a regex.
@picnixz
update NEWS
9e8fe5d
hugovk
hugovk approved these changes Jan 13, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@hugovk hugovk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@picnixz @hugovk
Update Misc/NEWS.d/next/Library/2025-01-10-13-06-54.gh-issue-118761.f…
5be6178 
…8oADD.rst

Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
@picnixz picnixz changed the title gh-118761: improve import time for base64 gh-118761: substitute re import in base64.b16decode for a more efficient alternative Jan 13, 2025
@picnixz picnixz requested a review from AA-Turner January 14, 2025 12:26
AA-Turner
AA-Turner reviewed Jan 14, 2025
View reviewed changes
picnixz and others added 2 commits January 14, 2025 13:43
@picnixz @AA-Turner
Update Doc/whatsnew/3.14.rst
5562c81 
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
@picnixz @AA-Turner
Update Misc/NEWS.d/next/Library/2025-01-10-13-06-54.gh-issue-118761.f…
82d18d7 
…8oADD.rst

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
@picnixz picnixz mentioned this pull request Jan 14, 2025
Closed
@picnixz picnixz requested review from AA-Turner and hugovk January 14, 2025 13:19
@AA-Turner AA-Turner merged commit bbd3300 into python:main Jan 14, 2025
@AA-Turner
Copy link
Copy Markdown
Member

AA-Turner commented Jan 14, 2025

Thanks!

A

@picnixz picnixz deleted the perf/import/base64-118761 branch January 14, 2025 13:49
@vstinner
Copy link
Copy Markdown
Member

vstinner commented Jan 14, 2025

@AA-Turner merged commit bbd3300 into python:main Jan 14, 2025

FYI @picnixz was recently promoted as a core dev and so can merge his own changes.

@picnixz
Copy link
Copy Markdown
Member Author

picnixz commented Jan 14, 2025

(I don't mind others merging my PRs by the way)

@chris-eibl
Copy link
Copy Markdown
Member

chris-eibl commented Jan 14, 2025 •
edited
Loading

and reduce the import time of :mod:base64 by up to six times.

9.5 ms down to 5.7 ms is impressive but not six times - rather 60%?

@picnixz
Copy link
Copy Markdown
Member Author

picnixz commented Jan 14, 2025

hyperfine benchmarks also take into account interpreter's startup and so, while -X importtime only takes into account the time needed to import the module itself (hence the differences).

@chris-eibl
Copy link
Copy Markdown
Member

chris-eibl commented Jan 14, 2025

Sure, my bad. Importtime itself is of course six times faster.

Thank you all for the great work here in cPython!

@picnixz picnixz mentioned this pull request Jan 17, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AA-Turner AA-Turner AA-Turner approved these changes

@hugovk hugovk Awaiting requested review from hugovk

+1 more reviewer

@effigies effigies effigies left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

performance Performance or resource usage

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@picnixz @AA-Turner @vstinner @chris-eibl @effigies @hugovk








ApplySandwichStrip

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier!      Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

Fetched URL: https://github.com/python/cpython/pull/128736

Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy