Casing rules restart from scratch on each segment of nested/styled content

[eltos]

When using styled content (such as boldface, underline etc.) the sentence- and title-casing does not work as expected.

Here is an example that shows the issue(s) related to style transitions, e.g. change of normal to bold text

• Sentencecase treats a style transition as if a new sentence was started
• Titlecase capitalizes words like "a", "for", "and" etc. unexpectedly if they appear as the last/first word before/after a style transition

There are two more things I find unexpected here, though these are not caused by styled content and might be a topic for a separate issue:

• The lowercasing of "WASM" only for sentence-, but not for titlecase
• The uppercasing of "is" for titlecase

---

Typst code to reproduce

#import "@preview/decasify:0.9.0": sentencecase, titlecase

#set page(margin: 1em, width: auto, height: auto)
#set text(lang: "en")

#table(columns: (3cm, 19cm))[Source][
  decasify is a *typst WASM package* providing typst _functions for locale and #underline[style guide aware]_ text casing.
]

== Sentencecase
#table(columns: (3cm, 19cm))[Actual][
  #sentencecase[decasify is a *typst WASM package* providing typst _functions for locale and #underline[style guide aware]_ text casing.]
][Expected][
  Decasify is a *typst WASM package* providing typst _functions for locale and #underline[style guide aware]_ text casing.
][Actual w/o style][
  #sentencecase[decasify is a typst WASM package providing typst functions for locale and style guide aware text casing.]
]

== Titlecase
#table(columns: (3cm, 19cm))[Actual (styled)][
  #titlecase[decasify is a *typst WASM package* providing typst _functions for locale and #underline[style guide aware]_ text casing.]
][Expected][
  Decasify is a *Typst WASM Package* Providing Typst _Functions for Locale and #underline[Style Guide Aware]_ Text Casing.
][Actual w/o style][
  #titlecase[decasify is a typst WASM package providing typst functions for locale and style guide aware text casing.]
]

[alerque]

Thanks for the report. This is actually more than one issue:

First, there isn't support for automatically excepting all-caps acronyms in Decasify yet. And the reason is it's almost impossible to do right for my use case. My main use case is taking data that is ENTERED IN ALL CAPS TO START WITH and converting it to other cases. There is no good way to recognize and retain acronyms when doing that. There is the potential to keep them when starting with a mixed-case string though, so as an optional style I'd be interested in supporting that. Can you open a separate issue to track that?

Second your expectations with "is" just don't git the Daring Fireball style guide, but I'm happy to help implement extra style guides (AP, CMOS, NYT, etc.) that might treat that according to your expectations. Which style guide actually has rules for handling it the way you want it?

Now back to the issue of styled content in Typst. I don't know how it breaks down content, but it might not be possible to do this because it might not pass us the content with styling information intact at all. That being said I'm quite willing to support whatever it gives us a mechanism for handling, so I'll look into it a little. If you know of any other packages doing any kind of transformation of input that retain style info I'd love to check out how they are doing it.

[eltos]

Thanks @alerque for the fast response and for making this package in the first place. Indeed those were side notes, I have opened dedicated issues as requested. The main point here was to handle styled content, though I'm afraid I don't know how typst handles it internally.

[ertugruluyar-github]

What if we enter all data as titlecase with acronyms like "Title Case with ACRONYM" and when converting to lower or sentence case, the words that already written uppercase excluded?
Or when using this case funtions, if given a parameter called "excluded-words" to specify what we don't change the case? And this in this parameter's default values we can define some default values that language specific. And also, to give user flexibility, we can also give another parameter called "included" to give user to force some words affected by the function.
What do you think about that?

[eltos]

What if we enter all data as titlecase with acronyms like "Title Case with ACRONYM" and when converting to lower or sentence case, the words that already written uppercase excluded?
[...]
What do you think about that?

See #40

[alerque]

I had a quick gander at this problem. First let use a somewhat simplified MWE that has the hard bits of this problem without overstepping the scope of other unrelated issues:

#import "@preview/decasify:0.9.1": sentencecase, titlecase

#set page(margin: 1em, width: auto, height: auto)
#set text(lang: "en")

#let sample = [this sample *has bold* and _italic #underline[underlined]_ bits.]

#table(columns: (2cm, 9cm))[Source][#sample]

= Sentencecase

#table(columns: (2cm, 9cm))[Actual][
  #sentencecase[#sample]
][Expected][
  This sample *has bold* and _italic #underline[underlined]_ bits.
]

= Titlecase

#table(columns: (2cm, 9cm))[Actual][
  #titlecase[#sample]
][Expected][
  This sample *Has Bold* and _Italic #underline[Underlined]_ Bits.
]

Notably, the unexpected outputs here do prove we are getting each style segment separately and processing them one segment at a time and outputting them as the correct expected style. The issue is we don't have any contextual information about what came before or after and whether that text should be considered when evaluating casing.

So far I haven't found any precedent either in other packages for evaluating the parent or sibling contexts. If anybody can point that out somewhere I'd be glad to look into how they did it.

[eltos]

we are getting each style segment separately and processing them one segment at a time

Right, that's because regex showrules don't span across style segments. The following needs a more sophisticated implementation.

In decasify.typ:

#let titlecase(body, style: "default") = {
  show regex(".+"): it => titlecase-string(it.text, text.lang, style)
  body
}

PS: I think your reduced MWE does not cover the case where the last word in a segment is capitalised, such as "A" and "And" in my MWE.

[alerque]

that's because regex showrules don't span across style segments

What does? Does anybody know?

This will need a two part fix. We'll still need to process each segment separately so we can return the content wrapped in the same style segments, but we'll want to send the full context (or at least any relevant previous/next sibling segments) to the decasify library and the library needs to take that into account when processing.