> The only thing that remains is related to bug 2 and the RTL text auto-detection in VTE. I am yet to hear from Egmont on anything we can do in this regard.

Both the autodetection "on" and "off" values have pros and cons. I don't think either one is better per se than the other. One is better in some circumstances, the other is better in others.

For complete sentences that are of mostly in a single direction, occasionally with an embedded word in foreign directionality, usually autodetection is better.

For a list of items, some of which might be of a foreign directionality (e.g. "ls -1"), aligning all of them in the same way (autodetection is off) is the better.

For users who input some text in their native language, occasionally containing a foreign directionality text, autodetection might mess things up if that word happens to be at the beginning, and again, a fixed overall directionality (autodetection off), matching the main language, is presuabmly better.

When it comes to designing a graphical app or a webpage, these are decided on a case-by-case basis. I can't see how a terminal could magically _solve_ this and provide a solution that's good enough for everyone. It provides a _platform_ where apps can pick which of the two behaviors they wish to have.

To make it more complicated, terminal emulation is an utter mixture of English vs. one's native language. Some pieces of text are English by their nature, some apps are not (or not fully) translated to Arabic (Persian, Hebrew...), etc.

Also, there are multiple use cases to take into account. One using their system in Arabic and encountering quite a few English words, or fully English utilities, is one thing. One using their system in English and encountering a few embedded Arabic words is another.

--

I'm afraid at this point we don't have anywhere near enough data to justify flipping the default, even despite that admittently picking the default was a somewhat arbitrary decision from me, with my overall impression being that this current default behavior might be better for users on average. Since I cannot speak/read/write any of the RTL languages/scripts, the decision might have been biased towards pure LTR environments (i.e. not to have random lines which happen to contain an RTL piece of text be right-aligned). After months of research and work on the topic of RTL in terminals, I did not have a strong stance on this.

Technically, you can flip the default, or create that shell script snippet that does this. Would this be a good solution? I'm afraid not, it would just probably make things more complicated, as the implementation would diverge from the proposed standard, multiple implementations would diverge from each other, app developers wouldn't be sure what to do.

The topic needs further research. It should evaluate which behavior is better under which circumstances, taking into account a wide range of apps, use cases. It should study both when a basic LTR environment has scattered RTL words in it (including the case of dumping binary data), and when a basic RTL environment has occasional LTR words (and numbers etc.).

Very importantly, the decision should heavily take into account which pieces of software can be more reasonably expected to switch to the other behavior. Ideally the need of the simplest tools (e.g. cat) should be the default, since they are the ones that cannot reasonably switch and later restore the behavior. Apps that do formatting for the terminal (e.g. ls for multi-column listing) will probably do have to do some BiDi anyway to preserve the columnar layout even in case of mixed directionality filename, so it's fair to request ls to switch to its preferred mode of operation and later switch back. Shells with their powerful interactive line editing also belong to this group: they already do a couple of things when presenting the prompt and when starting to execute the command (e.g. toggle bracketed paste mode back-n-forth), asking them to toggle some BiDi mode (in order to have the best BiDi experience when editing the command line) and restore when launching the command is a fair game. Asking the simplest tools, like cat, head, grep etc., and ad-hoc utilities, to switch back and forth is not that reasonable at all, so if there's a strong tendency which mode these utilities prefer then most likely that needs to be the terminal's default.

It could be that my proposed default behavior was poorly chosen and the other one would've been the better choice for the terminal (although I see no strong evidence for this yet). If this is the case, the way to fix it is first to study and understand the topic thoroughly, then if there are compelling reasons to change the terminal's default in the proposed spec (rather than fixing the problem elsewhere, like in those particular apps, including the shell itself; or in some shell script gluing) then update the spec, and finally (the trivial bits) adjust the upstream implementation.

That is, at least if you expect a solution that will work across distros (and eventually across terminals that implement the spec) and not just result in an utter incompatibility chaos.

At this point I don't know enough to see how to continue, and I'm afraid you don't know enough either. We'd need to understand a broad aspect of utilities, their demands, and foresee the forthcoming steps of the entire ecosystem to judge what to do. In order to make the next step, we must be able to see the big picture quite a few steps ahead. You have no chance of winning a chess game if you're only thinking one step ahead, right?

--

To make this even more complicated: I stopped working on terminal emulators years ago and moved towards other hobbies. Unless someone somehow manages to convince me and I find time and motivation to get back to this (which is quite unlikely to happen), I most likely won't. I'm afraid you'll have to take this issue in your hands.

But at least please come up with reasons way stronger than "these particular couple of apps that I've tried would look better in the other mode". Please make it a strong case that I made a mistake and the other default would be much better. And please try to do it in a way that all distros and all future terminals (who care about this BiDi stuff at all) benefit too; do it in a way that developers aren't confused why Ubuntu behaves differently than other distros, thus having no clue what to do in their apps... Design the best overall BiDi experience, thinking multiple steps ahead.

My BiDi proposal has been out there for 4 years, and the implementation in VTE for 3.5 years. This very bugreport here from you is pretty much the first valuable feedback I received (other than when I explicitly asked for feedback from an RTL expert, and later on the Unicode mailing list). RTL communities recognized my work and the potentials within it way less than I anticipated. I suspect one of the reasons is that terminal emulation is such a complex platform, capable of supporting apps with vastly different behaviors, that adapting BiDi is so complicated that even those familiar with RTL don't really see the big picture here. Another reason I guess is that due its forever brokenness and the current state of apps, people tend to avoid RTL-speaking terminal-based apps and rather go for either graphical RTL apps or English language in the terminal. Yet another reason is that very few people see beyond their particular favorite distro, their particular favorite terminal emulator, and their particular frequently used couple of applications. It will take a long time for developers (both devs of other terminals and devs of terminal-based apps) to realize the potential in the platform I designed for better BiDi. My suggested default for autodetection, one of the weakest decisions in that work, might be the less fortunate one, but if efforts are taken to change that, it's important that they are firmly laid down and clearly understand the big picture.