XML Sorter
Results 1 to 18 of 18

Thread: XML Sorter

  1. #1
    SOH-CM-2020
    Join Date
    Jun 2005
    Location
    Aotearoa, New Zealand
    Age
    63
    Posts
    2,896

    XML Sorter

    Just found this wonderful tool. It sorts xml tags alphabetically. So it allows the effects.xml to be put in alphabetic order at the push of a button.

    https://xmlsorter.codeplex.com/

    Why is this useful? Because the large effects.xml usually has repeat elements which are identical - possibly a hangover from when CFS3 was originally being compiled.

    It is tricky to find duplicate effects because so many effects have a long and complex name, often differing from another effect name by one character.

    Once I've sorted the effects xml I use an xml editor like XMLEd or "Firstobject XML Editor" to view the file, because the window in the sorter is a bit small. It is much easier to spot the duplicates if "tree view" is enabled - Firstobject has this feature as no doubt do other xml editors.

    This exercise started when I re-found Major Magee's version of the ETO Effects xml which he has very kindly provided in another thread - don't remember which one. It has all his gunsights added. So to test the new sorter I used his xml file, and found initially a bunch of duplicate effects starting with the tag "bullet_impact_ ......". I suspect they are entries from the stock Effects.xml, which have ported over to the effects.xml file in other installs like MAW, RS, ETO etc.

    There are a couple of issues with tidying up you effects.xml file. One is that all the useful tags (eg. Daniel's effects, O-1 Drivers Effects) become useless as the contributor's entries are alphabetised along with every other effects entry. The other is that some of the duplicate entries have the same name, but different subgroups (ie Effect0, Effect1, Effect2 etc). So it becomes a matter of judgement as to which of the duplicate pair to keep.

    Presumably when two effects are called the same, CFS3 randomly decides which one to give effect to at the appropriate moment in the game (eg when a gas tank catches fire).

    All that aside, the XML sorter is a great tool and I should update the Knowledgebase sticky post about how to sort the effects.xml.

    happy flying,


  2. #2
    SOH-CM-2023 mongoose's Avatar
    Join Date
    Jun 2005
    Location
    Navigator, where are we?
    Age
    79
    Posts
    3,555
    Interesting! Worth a look see.

    Cato said "Carthaginem esse delendam"
    I say "Carthago iam diu deleta,sed enim Bellum Alium adhuc aedificandum est"

  3. #3
    Interesting, thanks.
    [SIGPIC][/SIGPIC]
    www3.telus.net/murrdaka/

  4. #4
    I'd advise against this kind of thing. I think of it as an effect disorganizer. Group effects usually are located with their individual sub effects, and this all but guarantees they'll be split up all over the file. Also, CFS3 doesn't randomly select from among the duplicate effects (I wish it did, I could do some really cool things with that). Off the top of my head I can't recall if it reads the file top to bottom or bottom to top (I think it's bottom to top) but it uses the first name it comes to. So the last duplicate entry (or the first one if I'm wrong) is the one used every time. I'm not sure how the sorter works, but if it changes the order the duplicate effects appear in, then you may get some unintended consequences - i.e. CFS3 now uses an old stock effect that was superseded by a newer better one, or an odd effect might appear in a group effect. If it's working fine, I wouldn't mess with it.

  5. #5
    If it aint broke, don't fix it!
    John
    (DR/ MAW/ ETO/ PTO Textures)

    Keep it coming!

  6. #6
    SOH-CM-2023 mongoose's Avatar
    Join Date
    Jun 2005
    Location
    Navigator, where are we?
    Age
    79
    Posts
    3,555
    As in all experiments; keep a back up!

    Cato said "Carthaginem esse delendam"
    I say "Carthago iam diu deleta,sed enim Bellum Alium adhuc aedificandum est"

  7. #7
    SOH-CM-2020
    Join Date
    Jun 2005
    Location
    Aotearoa, New Zealand
    Age
    63
    Posts
    2,896
    Quote Originally Posted by gecko View Post
    I'd advise against this kind of thing. I think of it as an effect disorganizer. Group effects usually are located with their individual sub effects, and this all but guarantees they'll be split up all over the file. Also, CFS3 doesn't randomly select from among the duplicate effects (I wish it did, I could do some really cool things with that). Off the top of my head I can't recall if it reads the file top to bottom or bottom to top (I think it's bottom to top) but it uses the first name it comes to. So the last duplicate entry (or the first one if I'm wrong) is the one used every time. I'm not sure how the sorter works, but if it changes the order the duplicate effects appear in, then you may get some unintended consequences - i.e. CFS3 now uses an old stock effect that was superseded by a newer better one, or an odd effect might appear in a group effect. If it's working fine, I wouldn't mess with it.
    Hi Daniel, Too true. You raise a number of good things to keep in mind. I like the idea that the file might be read up or down. Makes a change from the usual infuriating randomness of file access for textures by CFS3.

    But it doesn't answer the question why there are duplicates (many identical) in the first place? And one or two of the duplicates is a highly compromised incomplete effect so I wonder if some effects have (in effect) been disabled?

    I like the idea of an organised effects.xml.

    Plus some people like to know what is going on in their xml files (compulsive xml dabbler with hand up here ).

    By using backups (like Mongoose says), this tool may reveal some unwanted artifacts.

    The way I've sorted some duplicate entries Is I've kept the ones with dr_ prefixed textures as at least I know they are not the old stock effects.

    So anyway, I hope this tool will provide endless fun for compulsive tweakers.

  8. #8
    Duplicate effects come from the various reworked effects set that are out there. First there was just the stock one, then Fox did a set that redid a lot of them and added a few new ones, then Firepower did the same, then MAW, then ETO. The easiest way to combine them all was just that, which insured all the new effects were present, but also introduced the duplicates. This didn't cause any problems since CFS3 chooses the same effect each time. After all the effects I've experimented with, my personal effects.xml would be your worst nightmare. To be fair, sometimes it's my worst nightmare too! But to each his own, like James says, make a backup and organize away, keeping in mind the possible pitfalls and it might save you some grief.

  9. #9
    SOH-CM-2020
    Join Date
    Jun 2005
    Location
    Aotearoa, New Zealand
    Age
    63
    Posts
    2,896

    Error Message

    Hi, gosd tells me he strikes an error message when he opens an effects.xml file in XML Sorter.

    Reminds me to post about it, thanks gosd. I have struck this with several effects.xml files, but once you fix the file, the sorter runs fine. It is an easy fix. The first error message from XML Sorter relates to an effect in the effects xml, which has two entries "effects0=". The Sorter tells you which line the error is on. I simply change the second entry to "effects1=" and all is fine. There may be a second error too, but from memory it was fairly self-explanatory to fix in the effects.xml.

    Once the effects.xml is tidied up with these two minor changes, the sorter works very well. However I have not tested all effects xmls for all installs, so there may be one or two other generic bugs.

    One interesting thing I am finding with one of my tweaked xml files (in MAW), is that when I bomb a ship and it explodes, there is a deep flash in the water. This effect never appeared before, so I can only think that cleaning up the effects.xml has fixed this. It certainly makes bombing ships more interesting!! One other possibility is that Ankor's latest shaders have made the difference but I like to think it is a result of tidying up the effects.xml

  10. #10
    After using XML wrench I have found duplicate attributes in my effects.xml folder under “InitialDelay” but I do not know which value to delete. Can you have a look? Thanks.

    <Afterburner_rato ClassName="ParticleEffect" InitialDelay="19.5" EmitterShape="Sphere" EmitterRadius="0" EmitterLifetime="36.5" EmitterPosX="-.05" EmitterPosY="0" EmitterPosZ="0" ZBias="0.0" EmissionAngleHoriz="180" EmissionAngleVert="93" EmissionSpread="8" EmissionRate="800" EmissionRateVar="100" InitialDelay="0" InitialCount="0" MinDistance="30" MaxDistance="2500" StopMethod="0" ZSort="0" UprightSprite="0" RandomStartRotation="1" RandomRotationDirection="1" Lifetime=".08" LifetimeVar=".01" Speed="25" SpeedVar="3" MaxSpeed="600" MaxSpeedVar="0.0" Accel="0" AccelVar="0.0" AccelTime="0.0" Gravity="0" Drag="0" DragDelay="0.0" InheritVelocity=".96" Size=".2" SizeVar="0" MaxSize=".015" MaxSizeVar="0" GrowRate="0" GrowRateVar="0" RotationRate="100" RotationRateVar="50" FadeInTime=".03" InitialColor="233 237 237" InitialAlpha="0" Color="225 235 240" Alpha="40" FadeOutTime=".06" FinalColor="216 216 216" FinalAlpha="0" Texture="fireball_additive2.dds" BlendMode="Add"/>

    <rato_smoketrail_short_1 ClassName="ParticleEffect" EmitterShape="sphere" InitialDelay="20" EmitterRadius="0" EmitterLifetime="25.5" EmitterPosX="-.05" EmitterPosY="0" EmitterPosZ="-2.0" ZBias="0.0" EmissionAngleHoriz="180" EmissionAngleVert="93" EmissionSpread="0" EmissionRate="200" EmissionRateVar="0" InitialDelay="30.5" InitialCount="0" MinDistance="30" MaxDistance="320" StopMethod="0" ZSort="0" UprightSprite="0" RandomStartRotation="1" RandomRotationDirection="1" Lifetime=".35" LifetimeVar="0.6" Speed="0" SpeedVar="0" MaxSpeed="0" MaxSpeedVar="0.0" Accel="0" AccelVar="0.0" AccelTime="0.0" Gravity="0" Drag="0" DragDelay="0" InheritVelocity="0" Size="1.0" SizeVar="0.9" MaxSize="200.0" MaxSizeVar="3.0" GrowRate="10.0" GrowRateVar=".05" RotationRate="1.0" RotationRateVar="8" FadeInTime="0" InitialColor="255 255 255" InitialAlpha="300" Color="255 255 255" Alpha="300" FadeOutTime="10.0" FinalColor="255 255 255" FinalAlpha="400" Texture="SmokePuff01.dds" BlendMode="QuadSprite"/>

    <rocket_exhaust_rato ClassName="ParticleEffect" InitialDelay="20" EmitterShape="sphere" EmitterRadius="0" EmitterLifetime="25.5" EmitterPosX="0" EmitterPosY="0.0" EmitterPosZ="-1" ZBias="0.0015" EmissionAngleHoriz="180" EmissionAngleVert="0" EmissionSpread="30" EmissionRate="400" EmissionRateVar="0" InitialDelay="0" InitialCount="0" MinDistance="0" MaxDistance="2500" StopMethod="0" ZSort="0" UprightSprite="0" RandomStartRotation="1" RandomRotationDirection="1" Lifetime=".35" LifetimeVar=".03" Speed=".01" SpeedVar="0" MaxSpeed="1000" MaxSpeedVar="0.0" Accel="-1.5" AccelVar="0.0" AccelTime="1" Gravity="0" Drag=".001" DragDelay="2" InheritVelocity="0" Size="1" SizeVar=".3" MaxSize="30" MaxSizeVar="10" GrowRate="50" GrowRateVar="10" RotationRate="30" RotationRateVar="15" FadeInTime="0" InitialColor="67 47 41" InitialAlpha="180" Color="135 133 133" Alpha="100" FadeOutTime="3" FinalColor="135 133 133" FinalAlpha="0" Texture="cloudpuff.DDS" BlendMode="QuadSprite"/>

    <Afterburner_rato_blitz ClassName="ParticleEffect" InitialDelay="44.5" EmitterShape="Sphere" EmitterRadius="0" EmitterLifetime="55" EmitterPosX="-.05" EmitterPosY="0" EmitterPosZ="1.0" ZBias="0.0" EmissionAngleHoriz="180" EmissionAngleVert="93" EmissionSpread="8" EmissionRate="800" EmissionRateVar="100" InitialDelay="0" InitialCount="0" MinDistance="30" MaxDistance="2500" StopMethod="0" ZSort="0" UprightSprite="0" RandomStartRotation="1" RandomRotationDirection="1" Lifetime=".08" LifetimeVar=".01" Speed="25" SpeedVar="3" MaxSpeed="600" MaxSpeedVar="0.0" Accel="0" AccelVar="0.0" AccelTime="0.0" Gravity="0" Drag="0" DragDelay="0.0" InheritVelocity=".96" Size=".2" SizeVar="0" MaxSize=".015" MaxSizeVar="0" GrowRate="0" GrowRateVar="0" RotationRate="100" RotationRateVar="50" FadeInTime=".03" InitialColor="233 237 237" InitialAlpha="0" Color="225 235 240" Alpha="40" FadeOutTime=".06" FinalColor="216 216 216" FinalAlpha="0" Texture="fireball_additive2.dds" BlendMode="Add"/>

    <ratoblitz_smoketrail_long_1 ClassName="ParticleEffect" InitialDelay="45" EmitterShape="sphere" EmitterRadius="0" EmitterLifetime="54.5" EmitterPosX="-.05" EmitterPosY="0" EmitterPosZ="-.50" ZBias="0" EmissionAngleHoriz="180" EmissionAngleVert="93" EmissionSpread=".1" EmissionRate="200" EmissionRateVar="0" InitialDelay="30.5" InitialCount="0" MinDistance="30" MaxDistance="320" StopMethod="0" ZSort="0" UprightSprite="0" RandomStartRotation="1" RandomRotationDirection="1" Lifetime=".35" LifetimeVar="0.6" Speed="0" SpeedVar="0" MaxSpeed="0" MaxSpeedVar="0.0" Accel="0" AccelVar="0.0" AccelTime="0.0" Gravity="0" Drag="0" DragDelay="0" InheritVelocity="0" Size="1.0" SizeVar="0.9" MaxSize="200.0" MaxSizeVar="5.0" GrowRate="10.0" GrowRateVar=".05" RotationRate="1.0" RotationRateVar="1.0" FadeInTime="0" InitialColor="255 255 255" InitialAlpha="300" Color="255 255 255" Alpha="300" FadeOutTime="10.0" FinalColor="255 255 255" FinalAlpha="400" Texture="smokepuff05.dds" BlendMode="QuadSprite"/>

    <ratoblitz_smoketrail_short_1 ClassName="ParticleEffect" EmitterShape="sphere" InitialDelay="45" EmitterRadius="0" EmitterLifetime="54.5" EmitterPosX="-.05" EmitterPosY="0" EmitterPosZ="-.50" ZBias="0.0" EmissionAngleHoriz="180" EmissionAngleVert="93" EmissionSpread="0" EmissionRate="200" EmissionRateVar="0" InitialDelay="30.5" InitialCount="0" MinDistance="30" MaxDistance="320" StopMethod="0" ZSort="0" UprightSprite="0" RandomStartRotation="1" RandomRotationDirection="1" Lifetime=".35" LifetimeVar="0.6" Speed="0" SpeedVar="0" MaxSpeed="0" MaxSpeedVar="0.0" Accel="0" AccelVar="0.0" AccelTime="0.0" Gravity="0" Drag="0" DragDelay="0" InheritVelocity="0" Size="1.0" SizeVar="0.9" MaxSize="200.0" MaxSizeVar="3.0" GrowRate="10.0" GrowRateVar=".05" RotationRate="1.0" RotationRateVar="8" FadeInTime="0" InitialColor="255 255 255" InitialAlpha="300" Color="255 255 255" Alpha="300" FadeOutTime="10.0" FinalColor="255 255 255" FinalAlpha="400" Texture="SmokePuff01.dds" BlendMode="QuadSprite"/>

    <rocket_exhaust_ratoblitz ClassName="ParticleEffect" InitialDelay="45" EmitterShape="sphere" EmitterRadius="0" EmitterLifetime="54.5" EmitterPosX="0" EmitterPosY="0.0" EmitterPosZ="-1.0" ZBias="0.0015" EmissionAngleHoriz="180" EmissionAngleVert="0" EmissionSpread="30" EmissionRate="400" EmissionRateVar="0" InitialDelay="0" InitialCount="0" MinDistance="0" MaxDistance="2500" StopMethod="0" ZSort="0" UprightSprite="0" RandomStartRotation="1" RandomRotationDirection="1" Lifetime=".35" LifetimeVar=".03" Speed=".01" SpeedVar="0" MaxSpeed="1000" MaxSpeedVar="0.0" Accel="-1.5" AccelVar="0.0" AccelTime="1" Gravity="0" Drag=".001" DragDelay="2" InheritVelocity="0" Size="1" SizeVar=".3" MaxSize="30" MaxSizeVar="10" GrowRate="50" GrowRateVar="10" RotationRate="30" RotationRateVar="15" FadeInTime="0" InitialColor="67 47 41" InitialAlpha="180" Color="135 133 133" Alpha="100" FadeOutTime="3" FinalColor="135 133 133" FinalAlpha="0" Texture="cloudpuff.DDS" BlendMode="QuadSprite"/>

  11. #11
    Quote Originally Posted by Bravo/4 View Post
    If it aint broke, don't fix it!
    Can't say that too many times. Unless you are a coder stay out of there. Granted the code might be bloated and messy but it works.

  12. #12
    SOH-CM-2023 mongoose's Avatar
    Join Date
    Jun 2005
    Location
    Navigator, where are we?
    Age
    79
    Posts
    3,555
    I think if one backs up whatever one is playing around with, not too much an issue. Always good to see if tweaks are better!

    Cato said "Carthaginem esse delendam"
    I say "Carthago iam diu deleta,sed enim Bellum Alium adhuc aedificandum est"

  13. #13
    SOH-CM-2020
    Join Date
    Jun 2005
    Location
    Aotearoa, New Zealand
    Age
    63
    Posts
    2,896
    Hi gosd, You have PM. Just send the effects.xml to me and I will check it. Then I can send it back to you for use with the sorter.

    @ sceptics and doubters - the best way to use the tool is on copies of the effects.xml files that are outside of the game (so that you don't mix them up!). One good use for the sorter is that after getting the effects file from two different installs "alphabetised", I can run them past Winmerge. Winmerge is a great programme which can compare two (nearly) identical files, and highlight the differences. This is very handy for a long complex file. It allows for combining effects files easily. I plan to produce a "super effects xml" just for the hell of it, which can work across several installs.

    Just have to remember all the lines from the DL 109s, Jason's Japanese ships, and sundry weapons mods...

    Recently, I found that the original MAW effects.xml was missing some of the AvHistory effects, so some tracers where underwheliming ie mostly absent. So I am hoping the combined MAW/AvHistory effects file covers any inadvertent omissions.

    Hope you guys in the Northern Hemisphere are enjoying the warm weather - winter is now Down Under... hopefully will give me more excuse with the missus for staring at long lines of text in xml files..

  14. #14
    SOH-CM-2020
    Join Date
    Jun 2005
    Location
    Aotearoa, New Zealand
    Age
    63
    Posts
    2,896

    Update

    I put together a combined MAW/ETO/AvHistory effects.xml last night. It is not too big - I have bigger effects.xml files floating about IIRC.

    First step was to make sure that the xml files were valid using an XML editor - I use XML Wrench, which has a couple of buttons on the toolbar which makes the job easy.

    Then I "alphabetised" the files using XML Sorter.

    Then the two files (ETO and a combined MAW/AvHistory file) were loaded into Winmerge - It helps to have a wide screen to compare the two files side by side. Winmerge makes the comparison of effects, and the identification of gaps, really easy. Missing effects that you want can be copied into the relevant file is done with the click of the mouse using the "copy to left" or copy to right option.

    The exercise was quite interesting as I could see the evolution of some effects from stock, to MAW, through to ETO. For example the dust effects for troopers or vehicles tend to linger longer for MAW than for ETO, which makes sense as MAW is generally a drier environment. In the wash up, I did a quick comparo of stock MAW with AvHistory's 2008 effects.xml. There were not too many files added with the Av History version. A lot of their effects were prefixed "ac_".

    When testing the final product, CFS3 would not digest the combined effects.xml until I had run it through XML Wrench again - some of the text in a UTF-8 file does not show up in Notepad, and CFS3 posted errors from the Microsoft xml parser.

    I seem to remember people having problems with this in the past (xml error messages in CFS3, which could not be seen in Notepad) - running it through an xml editor saves a lot of heartache.

  15. #15
    SOH-CM-2023 mongoose's Avatar
    Join Date
    Jun 2005
    Location
    Navigator, where are we?
    Age
    79
    Posts
    3,555
    Sounds interesting. I suppose one has to have a set up to compare 2 files under the same circumstances to see which is better.

    Cato said "Carthaginem esse delendam"
    I say "Carthago iam diu deleta,sed enim Bellum Alium adhuc aedificandum est"

  16. #16
    SOH-CM-2020
    Join Date
    Jun 2005
    Location
    Aotearoa, New Zealand
    Age
    63
    Posts
    2,896

    Tools for Merging Effects xmls

    Quote Originally Posted by mongoose View Post
    Sounds interesting. I suppose one has to have a set up to compare 2 files under the same circumstances to see which is better.
    The proggy I use is called Winmerge - it seems to be able to compare all sorts of files of basic text type. If you cannot find it just PM me and I will zip up a copy. It is super useful for other files like comparing xdp files when trouble shooting, by comparing with a stock example or a known working version. Where there are differences, one of the good features is with a simple click you can select a highlighted sequence of text and transpose it to the appropriate place in the corresponding file. Makes things pretty quick, like major effects merges.

    I remember years ago there was a thread called Merging the Monsters, where Charlie talked about combining the effects used for Day It All Began, with Foxes Super Effects (or was it Firepower effects?). Such a process is much easier with the XML tools I've mentioned, combined with Winmerge.

    Luckily ETO reflects the best of so much of the early work by Fox, Charlie, Nanni, AvHistory so not much of that sort of thing is really necessary. Although much to Daniel's horror I can't resist playing around with this stuff

    As for comparing lines of effects to see which one is better (sorry Mongoose taking too long to catch your drift ), yes that would be tricky but in some cases you can see what is going on, eg those dust effects I mentioned, logical that they hang longer in MAW. Also IIRC in MAW the colour for the basic explosions like artillery shells is made more brown than the stock texture, to reflect the terrain in MAW. So its good to have some different effects for different places, I wonder what is done in RS? The MAW textures would look silly in explosions in PNG, Solomons or China.

  17. #17
    SOH-CM-2023 mongoose's Avatar
    Join Date
    Jun 2005
    Location
    Navigator, where are we?
    Age
    79
    Posts
    3,555
    Came across this while looking for something. Thought up your street!

    GETTING EFFECTS.XML INTO ALPHABETICAL ORDER

    By Oki, January 2007

    "I have done it using EXCEL and found about 100 double entries in my effects.xml (of about 2900)

    that´s what I did:

    1. open effects.xml with EXCEL
    2. the lines defining the effects are stacked in one column, one line per cell
    3. the first and the last row contain only the word "effects"
    4. block the cells with the effects lines (beginning with row 2, not including the words "effects") and sort in increasing order (data/sort, without heading)
    5. go through the rows from top to bottom comparing the entries, if two are identical delete one of them (careful, some are just very similar, but not identical)
    6. save as txt-file
    7. open edited txt-file and copy-paste txt into new xml document of a xml editor
    8. save as xml-file

    by doing so I got an edited effects xml-file in alphabetical order with no double entries....."

    Cato said "Carthaginem esse delendam"
    I say "Carthago iam diu deleta,sed enim Bellum Alium adhuc aedificandum est"

  18. #18
    SOH-CM-2020
    Join Date
    Jun 2005
    Location
    Aotearoa, New Zealand
    Age
    63
    Posts
    2,896

    OKI's guide is in the Knowledgebase Sticky

    Yes thanks Mongoose, thats the system I used to use. The thing is, it is limited to those who have MS Excel, and also it is a lot more fiddly than using an xml tool.

    The only problem with the xml Sorter is that some people may have difficulty with CFS3's xml parser, which doesn't like some xml artifacts at the beginning of the file, which other xml tools are happy with. The artifacts at the beginning of some xml files need to be deleted before cfs3 will read the effects.xml.

Members who have read this thread: 0

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •