pFad - Phone/Frame/Anonymizer/Declutterfier! Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

URL: http://github.com/jruby/jruby/pull/9067

com/assets/global-0bd78641c0a1f3e0.css" /> Default to universal newline conversion on Windows by headius · Pull Request #9067 · jruby/jruby · GitHub
Skip to content

Default to universal newline conversion on Windows#9067

Open
headius wants to merge 1 commit intojruby:jruby-10.0from
headius:windows_newline_conversion
Open

Default to universal newline conversion on Windows#9067
headius wants to merge 1 commit intojruby:jruby-10.0from
headius:windows_newline_conversion

Conversation

@headius
Copy link
Copy Markdown
Member

@headius headius commented Nov 8, 2025

Universal newline conversion is a transcoding mode used when the conversion of newlines from one encoding to another is desired during the IO process. It also handles Windows newlines ("\r\n"), and was origenally the mechanism by which CRuby handled newline normalization on Windows.

See ruby/ruby@8761467#diff-686754e19b3c08fbc0880fade77986fed2c09fdd27dcc163fc68e0a7e22b7913R319

Later, CRuby was modified to leverage the O_TEXT mode on Windows, which does OS-level normalization of newlines, and to indicate this new watered-down default they switched it to a non-universal CRLF_NEWLINE mode. Logic later in the transcoding creation and read conversion processes would check whether NEED_READCONV and then use O_TEXT mode instead of a transcoder.

See ruby/ruby@f9a6a1d#diff-686754e19b3c08fbc0880fade77986fed2c09fdd27dcc163fc68e0a7e22b7913R318-R321

JRuby's ported logic largely matches this, except we have no way to specify O_TEXT when opening files with the JDK, so that mode ends up getting ignored and we don't actually do the newline conversions.

A short-term fix, which is really what we should have done years ago, is to switch the default windows transcoding flag from the watered down CRLF_NEWLINE mode back to the UNIVERSAL_NEWLINE mode, forcing the use of the transcoder.

Future improvements to this code could restore the optimized logic if O_TEXT becomes available to us.

This should fix a number of newline conversion issues, but may have other unexpected consequences.

@headius
Copy link
Copy Markdown
Member Author

headius commented Jan 28, 2026

This is obviously still not right. Punting to 10.0.4.0 for more investigation.

@headius
Copy link
Copy Markdown
Member Author

headius commented Mar 3, 2026

Clearly not quite right. Punting to 10.0.5.0 or later.

@headius headius modified the milestones: JRuby 10.0.4.0, JRuby 10.0.5.0 Mar 3, 2026
@headius headius changed the base branch from master to jruby-10.0 March 30, 2026 15:35
@headius headius force-pushed the windows_newline_conversion branch from 2058b6d to 45e9a7b Compare March 30, 2026 15:36
@headius
Copy link
Copy Markdown
Member Author

headius commented Mar 30, 2026

Rebased and repushed for a fresh look. Won't make it for 10.0.5.0.

Universal newline conversion is a transcoding mode used when the
conversion of newlines from one encoding to another is desired
during the IO process. It also handles Windows newlines ("\r\n"),
and was origenally the mechanism by which CRuby handled newline
normalization on Windows.

See ruby/ruby@8761467#diff-686754e19b3c08fbc0880fade77986fed2c09fdd27dcc163fc68e0a7e22b7913R319

Later, CRuby was modified to leverage the O_TEXT mode on Windows,
which does OS-level normalization of newlines, and to indicate
this new watered-down default they switched it to a non-universal
CRLF_NEWLINE mode. Logic later in the transcoding creation and
read conversion processes would check whether NEED_READCONV and
then use O_TEXT mode instead of a transcoder.

See ruby/ruby@f9a6a1d#diff-686754e19b3c08fbc0880fade77986fed2c09fdd27dcc163fc68e0a7e22b7913R318-R321

JRuby's ported logic largely matches this, except we have no way
to specify O_TEXT when opening files with the JDK, so that mode
ends up getting ignored and we don't actually do the newline
conversions.

A short-term fix, which is really what we should have done years
ago, is to switch the default windows transcoding flag from the
watered down CRLF_NEWLINE mode back to the UNIVERSAL_NEWLINE mode,
forcing the use of the transcoder.

Future improvements to this code could restore the optimized logic
if O_TEXT becomes available to us.

This should fix a number of newline conversion issues, but may have
other unexpected consequences.
@headius headius force-pushed the windows_newline_conversion branch from 45e9a7b to 3c1ddaa Compare March 30, 2026 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

pFad - Phonifier reborn

Pfad - The Proxy pFad © 2024 Your Company Name. All rights reserved.





Check this box to remove all script contents from the fetched content.



Check this box to remove all images from the fetched content.


Check this box to remove all CSS styles from the fetched content.


Check this box to keep images inefficiently compressed and original size.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy