OpenBve+Class 323 does not support plugin DLLs on Linux
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
bve-train-br-class-323 (Ubuntu) |
Opinion
|
Wishlist
|
Unassigned | ||
openbve (Ubuntu) |
Opinion
|
Wishlist
|
Unassigned |
Bug Description
Binary package hint: openbve
Routes and trains distributed for the BVE Trainsim (proprietary) and OpenBve (in Ubuntu) railway simulators consist primarily of text-based configuration and model files, which in turn reference to PNG/BMP/JPG images and WAV/OGG sound files.
In addition, the BVE Trainsim version 4 format allows each simulation/scenario to be programmatically extended by a compiled plugin. The format defines that this plugin is compiled to a Win32/PE+COFF/i386 DLL file. OpenBve (when also running on Win32) attempts to be backwards-
Inputs to the plugin arrive every frame, giving the state of the train, and ".Beacon" updates (one-way transponder messages placed along the track) , and from keypresses submitted by the user (the keys: Insert/
These plugins are generally non-free, non-distributable, and it is very unusual that the source-code is available. In the case of the 'OS_Ats1' plugin (last release 1.3 in 2005), source code *is* available and I have had communications with the author of the plugin to investigate releasing the source code under a DFSG-free licence. Possibly the GPL (already confirmed) or ideally placing the plugin code in the Public Domain (tentatively confirmed). The 'bve-train-
http://
When a train developer chooses to request the additional services provided by a plugin, a file named 'ats.cfg' is included in the appropriate 'Train/' directory (eg. '/usr/share/
The code path used by OpenBve to load and link to these plugins is to first extract the (optional) plugin name from the 'ats.cfg' file, then to thunk from the portable C#/CLR code to and from the Win32 DLL via an intermediate proxy:
'OpenBve.exe' (CLR/CLI) <> 'AtsPluginProxy
The upstream binary distribution includes a non-portable binary compilation of 'AtsPluginProxy
if(Program.
"...plugins are not supported on operating systems other than Windows."
if (IntPtr.Size != 4)
"...plugins are not supported in 64-bit environments."
I can confirm that refining these two checks allows the OpenBve code base to load and use a natively compiled version of 'OS_Ats1.so', via (an also natively-compiled) 'AtsPluginProxy
http://
http://
Some of the patches were accepted and integrated. However, in the case of specifically removing/
Upstream have indicated they wish to see plugins continue to be distributed *only* as Win32/PE+COFFs DLLs. The argument, as I understand it, is that the selection of the format for images, routes files and plugins was chosen by BVE Trainsim 4 (a proprietary program) and that those chosen (binary/text) formats are supported only for backwards compatibility reasons.
A fear was presented that allowing other compiled formats of plugins (eg. Libc/ELF/armle) might lead to additional incompatibility issues; an MS Windows user might complain that a plugin compiled for some other processor+ABI type and called MyPlugin.so would not load (as a binary) on /their/ system. A newer release of OpenBve now includes a minimal check for an 'MZ' (DOS executable) header.
Technically, the solution for "making [source available] plugins work on Linux" is very easy (removal of the overly-zealous Platform check), however, politcally it is a very different issue. In essence, whether to achieve backwards-
In order to satisfy OpenBve's upstream's requirements and those of Debian Policy, it will been necessary to cross-compile any plugin source-code to an i386+PE DLL (build dependency on 'mingw32'). Then to load this foreign i386+PE DLL shared library it will be necessary to parse the PE headers. For platforms !i386 it will be necessary to use emulation.
Load leaf-node DLLs (library which do not then go on to import any further external references) is relatively easy; the classic example is the 'win32-codecs' package and the loaders in FFMPEG. However, upon analysis most of the reviewed plugins further import Win32 API functions; such as to read and locate their own configuration files. Implementations of these further imported functions must be provided and the plugin "format" provides no restriction on what can be used.
Long-term it appears possible to write (from scratch) aPublic Domain pure C# PE parser, a pure C# i386 user-space interpreter (x86 emulator) and pure C# stub implementations of any imported Win32 functions, as these are discovered. In addition, bundling a built-in emulator+loader would on Linux/* but also Win64/amd64, Win64/ia64 and WinCE/* and Mac OSX without additional dependencies; not of these platforms have plugin functionality either. This avenue of development will be extremely time-intensive and although a very small amount of work has already been done in this direction, it boils down to reimplementing a subset of Qemu and Wine in a new language (C#).
A shorter-term (quicker) option would be to just *use* Wine to parse and load plugins and then to (crucially) resolve any Win32 imports---as Wine has a fairly complete set of of Win32 API function implementations already. On !i386, Qemu can then be used in addition, to provide the dynamic binary translation from i386 to the native instruction set. It should be possible to write a loader Win32 .exe program (somewhat equivalent to 'AtsPluginProxy'); this could be started inside Wine and to communicate and receive requests over a TCP connection opened by the parent CLR/CLI OpenBve.exe instance. This would ensure that cross-binary and cross-instruction set data exchanges only happen using a clearly defined wire, rather than via the exchange of pointers to unmanaged in-memory arrays.
Such a solution would have to 'Depends: wine' an entail introducing a large download for those demanding plugin support (just to support a 10 kB plugin). It would however work on Win64/amd64 systems without Wine, where the equivalent backwards-
In summary; the availability of out-of-the-box BVE plugins on Linux is not a technical problem, but a legal/political one: (a) source code of plugins is not generally available, nor DFSG-free for packaging, and (b) OpenBve actively restricts plugins to only one specific binary format and CPU instruction set.
We have a partial solution for (a) in the form of the source code and support of the OS_Ats1 author; but for (b) more work still needs to be done to support the requirements of upstream.
Changed in openbve (Ubuntu): | |
importance: | Undecided → Wishlist |
status: | New → Confirmed |
Changed in bve-train-br-class-323 (Ubuntu): | |
importance: | Undecided → Wishlist |
status: | New → Confirmed |
Changed in bve-train-br-class-323 (Ubuntu): | |
assignee: | nobody → Janis Fehrmann (janis96) |
Changed in bve-train-br-class-323 (Ubuntu): | |
status: | Confirmed → Opinion |
assignee: | Janis Fehrmann (janis96) → nobody |
Changed in openbve (Ubuntu): | |
status: | Confirmed → Opinion |
Hello Janis: I notice that you've assigned this bug report to yourself. Please could you give an idea of how you're intending to work on making Openbve work with plugins on Linux?