BNE CD unsupported

[ASDF]

The BNE cd should be supported.

Support/Tomes/Tome.1 is maindat.war
Support/Tomes/Tome.4 is strdat.war

[Pali (pali)]

Nobody has time for it. If you want to implement this you can send patches.

You can use StormLib library [1] or program SMPQ [2] for extract data from MPQ archives (found on WAR2 BNE)

[1] - http://zezula.net/en/mpq/stormlib.html
[2] - https://launchpad.net/smpq

[ASDF (asdf22)]

As far as I know there is no need to extract MPQ files and all that is required is to use the files in the tomes folder of the BNE CD as I said. It only needs to search for two different filenames. I don't know what data Tome.2 and Tome.3 contains, but Tome.1 and Tome.4 should be enough (maindat and strdat).

[Pali (pali)]

You are welcome to write patch for warextract.c

[Pali (pali)]

Here are some notes for BNE cd support:

Working WAR2 archives on BNE cd:
support/tomes/tome.1 = MAINDAT.WAR
support/tomes/tome.2 = REZDAT.WAR
support/tomes/tome.3 = SNDDAT.WAR
support/tomes/tome.4 = STRDAT.WAR

Missing WAR2 archives:
SFXDAT.SUD
MUDDAT.CUD

SFXDAT.SUD contains voices and sounds in Wave format. These files are stored in subdirectory Gamesfx in MPQ archive War2Dat.mpq. This archiv is saved in MPQ archive install.exe in subdirectory files. (Maybe some sounds are in subdir Speech ?)

MUDDAT.CUD contains movies in SMK format. These videos are stored in subdirectory Smk in MPQ archive install.exe

Music:

BNE cd does not contains additional audio CD tracks. Music (which may be same as on Audio CD) is in Wave format, stored in MPQ archive install.exe in subdirectory Music (or Speech ?).

===

So for BNE cd support is needed code for extracting MPQ archives.

[Alphamule (jwhiteheadcc)]

https://answers.launchpad.net/wargus/+question/160101
Da**! Shame on me for not having done it already. I totally forgot and left it for years. Sorry guys. I'm suggesting these be merged and I can submit the fix so we can change the installation FAQ. ;)

[Alphamule (jwhiteheadcc)]

Here's my plan:
1) Find a tool with compatible license (or make one based on documentation - not going to reinvent wheel though unless there's major [legal/technical] reason) to extract the .WAR files from MPQ archives. CLI Umpqx should work although a linkable library is preferred. https://umpqx.googlecode.com/svn/trunk/README
2) Create the code mockup for detecting if it's a BNE or 'green'(BDP) or 'blue' (original) CD. The BNE has a folder called TOMES and file called Install.EXE. The original game had a different (smaller) filesize and missing units, compared to the expansion set. It would makes sense to allow a little leeway since in theory, internation versions would have different sizes. I need a size/checksum list to do this right. The mockup will just call a (currently) dummy script that just returns an error code for invalid data (half true, hehe). DO NOT post this as a patch until steps 3-5 are complete!
3) Copy the TOMES to a temporary working folder.
4) Extract the extra 2 .WAR archives with the MPQ tool.
5) Point the normal .WAR extraction code to the temporary working directory.

I'll ask Pali if umpqx is an acceptable implementation to have in dependencies. I'm currently looking at those 2 links to compare licenses and implementation differences. Being a Ubuntu-oriented site, we strongly prefer free over nonfree.

[Bradley Clemetson (brad-c6)]

Take a look at my universal extractor, I am looking at picking up all the blizzard games, I would be AWESOME if you would be willing to contribute :).

I have used stormlib to manage the mpq files and hashes to detect the game version. Then using libxml2 and a configuration xml on how to extract the files. It still needs a lot of work in the area of transcoding with FFMPEG. It's @ https://github.com/bradc6/StrataExtract at the moment.

[Pali (pali)]

@Alphamule:

See my first comment. For extracting data from MPQ archives, you can use my tool smpq from: https://launchpad.net/smpq
It is licensed under GPL v3. If GPLv3 is not acceptable for you, I can relicense it. This tool can do anything with MPQ archives (extract, append, create, delete files).

[Alphamule (jwhiteheadcc)]

Yeah, I realized that it was your tool after looking at the other. Haha, nice. Seems that StormLib is the standard now. I'll get it working with a smpq-based implementation, then. I was thinking that bnetd had lists of checksums but I believe it's only for the main executable? Nevermind, as we (probably) don't need to worry about it. The 2 extra data files are technically optional unless you want the full experience. No movies and MIDI music just isn't the same. ;)

[Alphamule (jwhiteheadcc)]

Oh heck, just got into it and realized that meant the unit sounds, not the music. Extracting the unit sounds and movie files works with my outline but you'd have to repack them as .WAR archives which step 5 would just undo anyways. Would be silly, so might want to change that part of the plan, hehe.

[Alphamule (jwhiteheadcc)]

OK, almost done find the equivalent filenames inside the .MPQ and .WAR archives. As a side benefit, I'll get a more complete 1.3x knowledge base for WarDraft. :P I can see why you didn't do this. It's not that hard, just tedious as time-consuming and people already could use the older discs.

[Alphamule (jwhiteheadcc)]

Sorted by filesize, the MUDDAT.WAR indexes and filenames in Install.EXE(MPQ):
6 5 Smk\Intro_M.smk
16 15 4 3 IntroX_M.smk HVict_M.smk
9 8 OVict_M.smk
17 HVicX_M.smk
18 OrcX_M.smk
14 Zepp_M.smk
2 Demn_M.smk
10 Ships_M.smk
0 Burn_M.smk
12 Turtle_M.smk
11 Blizzard.bik

With multiple numbers, later are smaller versions. IntroX_M.smk is virtually the same as HVict_M.smk but is larger.

The list for SNDDAT.WAR and Install.EXE(MPQ):
50 humvict1.wav
51 humvict2.wav
52 orcvict1.wav
53 orcvict2.wav
54 orcvict3.wav
55 hum1_1.wav
56 hum1_2.wav
57 hum2_1.wav
58 hum2_2.wav
59 hum3_1.wav
60 hum3_2.wav
61 hum4_1.wav
62 hum4_2.wav
63 hum4_3.wav
64 hum5_1.wav
65 hum5_2.wav
66 hum6_1.wav
67 hum6_2.wav
68 hum6_3.wav
69 hum7_1.wav
70 hum7_2.wav
71 hum7_3.wav
72 hum8_1.wav
73 hum8_2.wav
74 hum9_1.wav
75 hum9_2.wav
76 hum10_1.wav
77 hum10_2.wav
78 hum10_3.wav
79 hum11_1.wav
80 hum11_2.wav
81 hum12_1.wav
82 hum12_2.wav
83 orc1_1.wav
84 orc1_2.wav
85 orc1_3.wav
86 orc2_1.wav
87 orc2_2.wav
88 orc3_1.wav
89 orc3_2.wav
90 orc4_1.wav
91 orc4_2.wav
92 orc5_1.wav
93 orc5_2.wav
94 orc6_1.wav
95 orc6_2.wav
96 orc7_1.wav
97 orc7_2.wav
98 orc7_3.wav
99 orc8_1.wav
100 orc9_1.wav
101 orc9_2.wav
102 orc10_1.wav
103 orc10_2.wav
104 orc10_3.wav
105 orc11_1.wav
106 orc11_2.wav
107 orc12_1.wav
108 orc12_2.wav
109 orc12_3.wav

[Alphamule (jwhiteheadcc)]

OK, I've changed my mind. I actually think it's better to make a MoPaQ enabler tool that just recreates the .WAR files in the data folder. I should test the higher data-rate intros and movies on the DOS version. If they work, then we get two tools in one! :) Also, it's not exactly like it's a huge amount of temporary storage.

Pali, if you still think it's best to put it into warextract (which looks like just a stub to load Meimo?), then I could have one #include <mpqenabler.c> line to do this in the same executable. The alternative is to rewrite wartool to also support the BNE CD which looks a lot less manageable. It just seems cleaner to not have to rewrite an almost exactly identical copy of most of wartool's source, just to look at BNE CDs. Then every time you fix one, you have to worry if maybe the same change needs to be done to the MPQ path. Or maybe I'm "doing it wrong" and there's some better way.

[Pali (pali)]

If you want to add support for MPQ archives, it should be in wartool binary. I already moved/rewritten all other programs/scripts into wartool, because I wanted to have one binary which extract data (without needs to run other binary or script which do something...). Of course you do not need to put functions into wartool.cpp file. You can add new functions into new cpp files and just call then from wartoot.cpp. Or you can also call external programs from wartool (if it is not possible to include code just to wartool binary). But make sure that wartool will extract & covert all data without need to run other script/program manually.

Other solution is to add MPQ and needed data support to Universal Extractor, but I do not know state of this project...

[Alphamule (jwhiteheadcc)]

OK, the quick hack which I fell asleep in the middle of remaking a WarDraft-like CLI tool for would work but I had to take a couple days break from the project. Yeah, best solution in long run is make a War+MPQ combination tool but I figure the war-archiver will be useful, too. The format is incredibly basic like PKZip. It just has some magic bytes, real basic sanity checks (number of files, etc.), and a table of file positions and lengths. The real gotcha is that files below 5 bytes or so get stored directly. I don't think I've seen any files that small besides 1-byte files.

BTW: If you REALLY want to see the old WarDraft documentation on the format, you have to go to help in the program. It's not in a text file but the help file.

[Alphamule (jwhiteheadcc)]

Yeah, I guess the MPQ detection routine would obviously need to go into wartool.cpp. I'm trying to figure out how to make it work with both MPQ and WAR without having to essentially repeat half the code. I think I might end up having to create an index/map between MPQ filenames and WAR entries, anyways. :/

[cybermind (iddqd-mail)]

After so many years I made some basic support for BNE in rev.1824 (no videos and unit sounds yet)

[cybermind (iddqd-mail)]

And after a while, this is completely done in rev.1859