pygettext: use an AST parser instead of a tokenizer #104400
Description
Follow up on this forum discussion
This is a part 1/X of improving pygettext. Replacing the tokenizer that powers the message extraction with a parser will simplify the code (no more counting brackets and f-string madness) and make it much easier to extend it with new features later down the road.
This change should also come with a healthy dose of new tests to verify the implementation.
PR coming shortly ;)
Linked PRs
- gh-104400: pygettext: use an AST parser instead of a tokenizer #104402
- gh-104400: Add more tests to pygettext #108173
- [3.13] gh-104400: Add more tests to pygettext (GH-108173) #126361
- [3.12] gh-104400: Add more tests to pygettext (GH-108173) #126362
- gh-104400: Remove
fintl.gettextfrom pygettext #129580 - gh-104400: pygettext: Prepare to replace TokenEater with a NodeVisitor #129672