Project file should be a plain text file

[markba]

The OpenShot project file should be a plain text file. It is now a binary, thus not human readable nor adjustable.

Advantages of using clear text:
- you can clearly see what and how things are organised
- things can be easily adjusted when something goes wrong
- you can do automated text operations with grep/sed/awk etc.

This is related (technically) to the problem where the path of the thumbnails of imported clips are registered in an absolute way. Moving the project file to another location or renaming the folder containing this project file, leads to not properly display of the thumbnails within the clips on the IDE. This could be adjusted, as a workaround, if the project file was in plain text.
https://bugs.launchpad.net/openshot/+bug/423948

[markba (mark-baaijens)]

Openshot 0.9.54
Ubuntu Jaunty 9.04

[Diode (jmail2)]

Excellent suggestion.
Project files from Videopad are human-readable with lines like :

hindex=19217
folders=0
files=55
clips=28
h=3&hfile=2&hfolder=0&name=00773.MTS%20-%20Clip%201&in=0&out=26320
h=6&hfile=5&hfolder=0&name=00774.MTS%20-%20Clip%201&in=0&out=47640
h=9&hfile=8&hfolder=0&name=00775.MTS%20-%20Clip%201&in=0&out=36200

- the URL encoding makes it a bit tricky but people with messed up projects have been able to rescue the situation. It would fit well into the FOSS ethos to make it clear and editable. I can't say I'd be keen on XML though :)

[hva (francesco-hermanitosverdes)]

I made some changes that add the possibility to save a project-file in ascii mode rather than binary.
The choice is set through preferences window.
Saving in ascii mode permits to modify some details of the project using a text editor (e.g. replace clip files
references)
As I've tried to implement it now this feature simply saves in ascii using cPickle serialization ascii protocol,
in the future it could be used to save in other forms (e.g. xml).
Added some effects XML files as well.

[hva (francesco-hermanitosverdes)]

Sorry, the previous diff introduced a bug due to a missing parenthesis.
here is a cleaner and more correct patch

[Jonathan Thomas (jonoomph)]

Thanks for the patch! I will take a look at it soon. =)

On Mon, Jan 11, 2010 at 12:04 PM, hva <email address hidden>wrote:

> Sorry, the previous diff introduced a bug due to a missing parenthesis.
> here is a cleaner and more correct patch
>
> ** Attachment added: "openshot_save_as_ascii.diff"
> http://launchpadlibrarian.net/37728919/openshot_save_as_ascii.diff
>
> --
> Project file should be a plain text file
> https://bugs.launchpad.net/bugs/499733
> You received this bug notification because you are a member of OpenShot
> Developers, which is subscribed to OpenShot Video Editor.
>
> Status in OpenShot Video Editor: New
>
> Bug description:
> The OpenShot project file should be a plain text file. It is now a binary,
> thus not human readable nor adjustable.
>
> Advantages of using clear text:
> - you can clearly see what and how things are organised
> - things can be easily adjusted when something goes wrong
> - you can do automated text operations with grep/sed/awk etc.
>
> This is related (technically) to the problem where the path of the
> thumbnails of imported clips are registered in an absolute way. Moving the
> project file to another location or renaming the folder containing this
> project file, leads to not properly display of the thumbnails within the
> clips on the IDE. This could be adjusted, as a workaround, if the project
> file was in plain text.
> https://bugs.launchpad.net/openshot/+bug/423948
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openshot.developers<https://launchpad.net/%7Eopenshot.developers>
> Post to : <email address hidden>
> Unsubscribe : https://launchpad.net/~openshot.developers<https://launchpad.net/%7Eopenshot.developers>
> More help : https://help.launchpad.net/ListHelp
>

[markba (mark-baaijens)]

Having looked at the patch, I have a few remarks.

1. The detection on what type of project file should be automatic. I suggest something like this (in pseudo-code):

try
  [open project file as text]
  OpenProject()
except
  try
    [open project file as binary]
    OpenProject()
  except
    [launch error message: "Could not open project file"]
  end
end

This ensures backwards compatability. This cannot be done with a global setting in preferences (as used in the patch), because one is never sure which type it is, especially when the user has switched his preference.

2. The default format for new projects should be text.

3. IMO, there's no need in putting the format as a user preference, because storing as a binary serves no real advantage, other then storage size; size is normally not a problem, considering the size of the resources (clips audio/video).

[hva (francesco-hermanitosverdes)]

cPickle doesn't seem to make any difference in de-serializing a binary or text format: it seems to get recognized without problems, so there's no need for a try-except routine in opening the file, I think.
If you set "ascii" as your user preference, you can open a binary project file with no apparent problem (and vice-versa), and when you re-save it it will be saved in ascii mode.
Anyway, it seems to work, but I intended it mainly as a workaround, I'm not sure it doens't give problems in some occasion.
WHat I can say is that it helped me in tranfering a project folder, as I saved in ascii format and then edited the file to fix the paths, and it worked.
Maybe in the long run it would be more flexible to introduce an XML based project file format.

[markba (mark-baaijens)]

OK, thanks for the explanation.

So it really comes down to making the default for new projects 'text', everything else will be managed automagically. This new default can also be taken when saving a project: when opening a project as binary, this will be automatically saved as text, doing the conversion on the background and without programming.

That will all be great I guess. Minimal programming, maximum output. We're making progress, and fast!

[Jonathan Thomas (jonoomph)]

There is now a preference for the user to change between binary or text project formats.

[Narcis Garcia (narcisgarcia)]

Text format should be the default.
Rarely the user will need binary format.

[moimael (moimael)]

I agree, and text format should be the only available.

It add useless complexity to allow the user to choose between project format.

I think the best thing to do is to made Text format defaut (binary do not have any advantages), not allow the user to change it and detect by code wich format is loaded for backward compatibility.

[okor (jason-ormand1)]

Text only is the way to go, for sure.

[D. Marxsen (spamfi)]

I would love to have the format of the project file documented.
This is because I would like to generate a project file from scratch via another application based on certain raw media files.

[moimael (moimael)]

I agree, the project file format should be documented !

[Thangalin (thangalin-deactivatedaccount)]

You might want to consider using XML for the file format. This would easily allow for:

- File format version (useful for reading/upgrading old file formats)
- Character encoding (file names with accented characters + ASCII = failure)
- Schema-based validation (point to an XSD and your work is practically done)
- Parallel development of tools and scripts for modifying the files independently of OpenShot (thus avoids introducing bugs into OpenShot)
- Logical hierarchy that is partially self-documenting
- Leverage XPath expressions

For example:

<?xml version='1.0'?>
<openshot version='1.1'>
  <project-files>
    <file-list id='0001' path='/home/user/projects/videos/my-video/audio'>
      <file id='000001' name='battle-royale.mp3' />
      <file id='000002' name='enter-knight.mp3' />
    </file-list>
    <file-list id='0002' path='/home/user/projects/videos/my-video/images'>
      <file id='000001' name='battle-scene.png' />
      <file id='000002' name='castle-gate.png' />
    </file-list>
  </project-files>
  <track-list>
    <track id='1' name='Background'>
      <file-ref id='01' list-id='0001' file-id='00001' start='00:00:04.000' end='00:00:05.500'>
        <animate ... />
        <transition ... />
      </file-ref>
    </track>
  </track-list>
</openshot>

You get the idea.

[Eric Prud'hommeaux (ericprud)]

If you extend SMIL¹, you get a bit of interop with other video engines (e.g. RealPlayer, HelixPlayer², Ambulant³) and a specification which has had some pretty thorough peer review (including patent safety). Inkscape did this with SVG with I believe good results. SMIL covers the animations; you'd have to extend it to include the catalogue info you've described above. Following is a brief example of SMIL animation.

¹ http://www.w3.org/TR/REC-smil/
² https://player.helixcommunity.org/
³ http://www.ambulantplayer.org/

<!DOCTYPE smil PUBLIC "-//W3C//DTD SMIL 2.0//EN"
     "http://www.w3.org/2001/SMIL20/SMIL20.dtd">
<smil xmlns="http://www.w3.org/2001/SMIL20/Language">
  <head>
    <meta name="title" content="For OpenShot" />
    <layout type="text/smil-basic-layout">
      <root-layout height="768px" width="1024px" backgroundColor='white' />
      <region id="r1" top="20px" left="0px" height="720px" />
      <region id="credits" top="768px" left="380px" height="1200px" />
    </layout>
    <transition id="snake" dur="1s" type="snakeWipe" />
    <transition id="clockR" dur="2s" type="clockWipe" direction='reverse' />
    <transition id="iris" dur="2s" type="irisWipe" />
  </head>
  <body>
    <par title='Intro'>
      <img src="background.jpg" erase="never" />
      <seq begin='5s'>
        <img src="ready.png" dur="5s" region='r1' />
        <audio src='woza.wav' begin="2s" title='Intro music'/>
      </seq>
      <seq begin='14s'>
        <img src="well_txt.png" dur="6s" transIn='snake' transOut='snake' />
        <par>
          <img src="welcome.jpg" dur="15s" transIn='iris' region='full_image'/>
        </par>
      </seq>
    </par>
    <par begin="4s" title='Credits'>
      <audio src='edith.wav' title='Intro music'/>
      <seq>
        <img begin='5s' region='credits' src="cred.png" dur="83s" fill='remove'>
          <animateMotion targetElement='credits' to="380,-1200"
                         begin='2s' dur="80s" fill='freeze'/>
        </img>
        <img region='r1' src='cont.png' dur='10s' transIn='clockR'/>
      </seq>
    </par>
  </body>
</smil>

[Jonathan Thomas (jonoomph)]

The default project file type is now ASCII instead binary. However, this will only affect new installs of OpenShot, or if a user removes their ~/.openshot/ folder. As a work-around for current users, they can just change the project file type in the preferences.