absolute paths in .osp

[Dmitry]

Hi,

I have opened .osp file with text editor and seems like paths to media files in project are absolute.
It will be great to use relative from .osp file paths to media files.

It should be much easy to move(send) clip sources and project file to another location

[Anonymouse (anonymouse2048)]

hmm... Maybe an option (albeit rather size-increasing) to pack the original .osp file along with all the data files into an archive, which still includes the file paths (allowing updates without reimporting) and if they exist, the files found on the hard drive are used. Otherwise the files packed into the archive are used.

Just an idea :)

[Ulrik Mikaelsson (rawler)]

At the very least, there should be some option to relocate a file-link that has been broken, by pointing out it's new location.

And really, yes, if the file is close to the OSP, the link should be relative.

[Leonidv (leonidv)]

Is not good, that files stored as absolute files. Really I can't move OpenShot project to another computer easy. More better is save name relativity by OSP file. Then I'll can make this structure of folder with project:

my-project/
source/ with source video and music files;
output.avi
my-project.osp.

OSP will contain file as:
source/1.avi source/2.avi etc.

May be it can options for project - relative or absolute pathes in OSP.

[Christian Weiske (cweiske)]

Scribus has an option to "collect external files". It simply copies all linked files in a selected folder and makes them relative in the project file afterwards. This could also be a solution for the OpenShot problem.

[Alfredo Rubio (alfredx8)]

I prefer the relative path, like in html, so there's no problems when moving files to another place.

[Leho Kraav (lkraav)]

i am also finding out now that it is impossible to copy my first bigger OSP project from laptop to my quadcore server for exporting, because paths don't match. quite a showstopper.

[Erik1984 (erik1984)]

I can confirm this. To make things worse it saves files to my desktop by default. I opened the .osp files with Gedit and replaced all occurrences of /home/user/Desktop with my desired new location and moved all folders and the respective .osp file there.

[anatoly techtonik (techtonik)]

I would raise the importance of this bug from Wishlist into something more plausible. Without this feature it is impossible to move project files to other place or a flash drive.

[battmanux (battmanu)]

Could anyone try this?
The basic idea is that when you open a .osp project, we check if project location has changed. If it is the case, we check if resources location have moved with the project or if they are still in the original location.
Special care is taken for files with %d in the name (image sequences).*

Emmanuel

classes/open_project.py line 74 (in bzr 1.4.3alpha1)
============USE===============
        #check project files still exist in the same location
        missing_files = ""
        items = project_object.project_folder.items

        # Just in case the project was moved
        new_project_path = "/".join(file_path.split('/')[:-1])

        for item in items:
            if isinstance(item, files.OpenShotFile):
                # Try auto path recovery in case the project was moved: Assume that resources were moved too, preserving relative paths
                if project_object.folder != new_project_path:

                    # project moved. Update files if resources moved or disapeared from prevouse path
                    if item.name.startswith(project_object.folder):
                        # File could be relative path
                        new_path = item.name.replace(project_object.folder, new_project_path)
                        if check_if_file_exist(new_path):
                            # The file exist relatively to the new project location. Update path
                            item.name = new_path
                        elif not check_if_file_exist(item.name):
                            # The file disapeard from previouse location. Considere that it was
                            # intended to be moved with the project file but relative new path is wrong.
                            # Update path
                            item.name = new_path
                        else:
                            # The file is not in the new location, and still in the old one.
                            # Keep the old one. This would generate absolute path.
                            pass

                    else:
                        # This file was absolute path anyway
                        pass

                if not check_if_file_exist(item.name):
                    missing_files += item.name + "\n"

        # If project has moved since last edition: update internal path
        project_object.folder = new_project_path

        if missing_files:
            messagebox.show("OpenShot", _("The following file(s) no longer exist.") + "\n\n" + missing_files)

============== additional function =================

def check_if_file_exist(item_name):
    """Same idea as os.path.exist, but handle openshot special filenames"""
    if "%" not in item_name:
        return os.path.exists(item_name)

    # We have a sequence file with %d in the path
    path = os.path.dirname(item_name)
    file_name = item_name.split(os.path.sep)[-1]
    head = file_name.split("%d")[0]
    tail = file_name.split("%d")[1]

    if not os.path.exists(path):
        return False

    for f in os.listdir(path):
     ...

[lale (lalelale)]

I understand the reasoning behind the patch by battmanux (I could not yet try it).
However I would discourage adopting such a complicated behavior in which openshot tries to "fix the project itself" without asking -- just consider what confusing behavior this would have for users who have source video files with the same name in different locations of the disk.

In my opinion, at least at first it would be safer to adopt one of the following, well-defined behaviors.

Option A) Just use relative paths for everything
Option B) Use relative paths for anything within or below the .osp file directory, keep absolute paths for everything else (similar to what PowerPoint does with linked video files: if the linked file it's in the same directory as the ppt it's expected to be moved around together with the ppt, else it's an absolute path).

Just my 2 cents

[battmanux (battmanu)]

I fully agree with the fact that doing magic without telling the user could be worse than doing something simple but well understood.
I would personnaly prefere your option B. It is actually what I first tryed as a fix. I found it to be not enough in my 'personal use case':
I save ma camcorder files in ~/Video/MyCam/[Year-Month]/
I create a project file in Openshot and save it in the default place (~/)
My project goes from 'little test' to 'big project'
I realise that generated files are filling my ~/ folder and thus,
I decide to move the project and ~/ related files in ~/Documents/OpenShot/July_Vacation/
My source video stayes in ~/Video/MyCam/[Year-Month]/
However: ~/Video/MyCam/[Year-Month]/ is a subfolder of ~/
This was resulting in breaking most filepaths...

That is the story of how I came with the above 'magic'. It is just what I was expecting to do by moving the project:
1/ If file exist in the new location, it means I did something to have it there while moving the project -> use the relative path
2/ If file does not exist in the new location, but is still in the old one, this is an obsolute path -> keep the old one
3/ If file disapeared but was in a subfolder of the original project, assume that is is a manipulation error in the process of moving relative path files while moving the project. -> use the relative path

My understanding of your 2 cent input is that case 3 shall be checked against the user before applying. Would this it be enough to you?

An other point would be to add a 'collect files/fix path' and copy everyting in a normelized subfolder tree.

Tell me if you see something else,

[lale (lalelale)]

I understand, and I think that your solution would be way better than the current situation.

BTW another principle would be to always use a dedicated directory for each (group of) open shot project(s). The rationale is quite simple: an osp almost never exists alone, but most often spawns additional files (e.g. title SVGs, outputs, etc...). Therefore it really makes little sense to leave in a generic directory (like ~).

However, one exception is the mentioned group of related projects. For example, different cuts of a video (a long and a shortened version), or different parts of a long video which would be handled through different projects. In this case, it makes sense that the osp files live in the same directory, so that they share the files they depend on.

[battmanux (battmanu)]

What about this:
When opening project, if project path has changed, warn the user and ask if he want to do a manual or automatic file path update.
Automatic is what is described above,
Manual open the 'collect files/fix path' window (to be designed)

Basic function offered by the 'collect files' window would be:
For each file referenced by the project:
1/ Displays last known path,
2/ Allows to handly (over)write file path,
3/ Sugests new location (normalized subfolder, updated relative path, original path, ...)
4/ Displays GreenCheck/RedCross icon whether file exist or not,
5/ Checkbox for 'copy' / 'transcode' file from last known path to new path (active only if file exist in last path but not in new path)

Pint 5 might be out of topic but such an option would help me to work on HD footage using my laptop. I could easely transcode footage I am using to a 'low quality/easely decodable' format and revert back to original HQ before rendering.
Step1: Import video files from absolut path location
step2: 'collect files' -> 'copy' with the new location in a 'HighDef' subfolder
Step3: 'collect files' -> 'transcode' with the new location in a 'LowDef' subfolder
Step4: for each new file added, repeate step2 and 3 if needed
Step5 (before rendering/fine tuning): 'collect files' -> manually replace 'LowDef' for 'HighDef' in file paths

I do not know if something similar is planed for futur releases.

[lale (lalelale)]

Sounds like a super-useful addition. :-)

[ggallozz (ggallozz-gmail)]

let see.... btw take it quickly

[battmanux (battmanu)]

check first step of the fix here https://code.launchpad.net/~battmanu/+junk/openshot

[markinfo (marek-straka)]

Why is this after 2 years only in wishlist? Why was originally used absolute paths at all? I am syncing my data over more machines and it makes troubles.

[ggallozz (ggallozz-gmail)]

+1 for Marek, WHY wishlist? I't's a main stuff, wnd not only for someone....

Let's go on!

[battmanux (battmanu)]

It seems that my first part of fix for this is not merged in 1.4.3. If you apply the diff to 1.4.3 you would get it (or just replace the file openshot/classes/open_project.py as there is no other changes in the file)
http://bazaar.launchpad.net/~battmanu/+junk/openshot/diff/683

Now, I may have missed something as I have been assigned for this bug and I do not know if it gives me more power to merge a fix or not.

However I assume that the modification has not been merges as it is a 'first part' and we may want the entire fix before committing.
This first part simply tries to solve path after a project being moved from one directory to an other. The second part shall provide a way to edit the location for each file and, why not, permit to 'on demand' swap between low and high quality version of the same shot.
I am sorry that I did not have enough time to contribute and to get the full feature out on time for 1.4.3.

[battmanux (battmanu)]

I moved my branch to 1.4.3. if you need/want to check the fix, pull http://bazaar.launchpad.net/~battmanu/+junk/openshot

[Jennifer (jennifer-uncharted-worlds)]

I've just run into this. +1 for relative addressing please :-)

I'm not sure of the progress of battmanux's patch now, but assuming we don't have actual relative addressing yet, meanwhile I say +1 to Ulrik's comment #2: "At the very least, there should be some option to relocate a file-link that has been broken, by pointing out its new location." I've only got about a dozen files as part of my current project, so doing it manually would have worked for me.

= Thoughts on use cases =

My example right now is that I started something on my "usual" computer but I'm finding this machine isn't powerful enough to run the video smoothly. The next bit is going to involve some speech sync-ing which I shan't be able to check with such choppy video, so I thought I would move it over to a friend's machine which is faster.

Apparently I can't. But I also can't finish the project on this machine. :-(

Another likely case is backups and archiving: would I be right in thinking that at present you'd have to ensure that any future machine has exactly the same names for its discs as your current one, and same directory structure, or your old projects become un-openable?

= Documentation =

This limitation is non-obvious until you run into it (and non-trivial to be caught out by). Therefore I suggest it justifies flagging in the manual.

I'd like to suggest adding a line to the "Projects - Create, Open, and Save" page: e.g. "Please start your new project on the same computer you will finish it on, [or on portable storage,] as Projects cannot yet be moved off one computer and onto another. The files you import to the Project will not be stored as part of it, and they must stay in the same place in their directory system." (if that's an accurate summary.)

If _I'd_ read that, I'd have either put it on a memory stick in the first place (if that worked), or at least not continued beyond the point where I started thinking "I'm going to need a faster machine to finish this off". (I did realise from experience of other programs that the clips might not be saved as part of the project file - but it never occurred to me that the solution wouldn't simply be to copy them across together with it.)

I understand that coding new bits can take much more time than lookers-on might imagine, but the time it takes to warn people in the Help must surely be small.

Thanks for reading if you got this far :-)

[battmanux (battmanu)]

thanks for the use case Jennifer,
what version are you using / what distrib ?

[battmanux (battmanu)]

how to patch openshot 1.3.4 yourself:
open a terminal and type:
$ cd
$ wget http://bazaar.launchpad.net/~battmanu/+junk/openshot/diff/683
$ locate open_project.py
answer shall be /usr/share/pyshared/openshot/classes/open_project.py
$ cd /usr/share/pyshared/openshot/classes
$ sudo patch open_project.py < ~/683
That's it,
Enjoy

[battmanux (battmanu)]

Other tip:
.osp files are 'kind of plain text' files.
If you are really stuck, you can try to edit it with gedit or other and do search/replace for the video file path that has changed.
make sure you keep everything else as is and do not have special char in modifications you are doing. And of course, backup your osp file in a safe place before modifying it!!

[Valio (valio-visionsinteractive)]

This should be *really* fixed.
For me it would be already enough if I could edit the absolute into a relative path manually. But not even that works; even for the internal project folder "thumbnail". It's completely bogus to prohibit using relative paths.

I would suggest to do the implementation priority as follows:
1. Let OpenShot accept OSP files who were edited manually with relative paths.
2. Assure that files from an OSP that had a relative path, keep a relative path, when saving changes (and vice-versa).
3. In the UI add a switch in "preferences" (or in the settings of a "projects file") in order to let the user by GUI determine whether the path should be stored absolute or relative for the files.
4. Fancy optional stuff: do something like in InDesign, where on opening a project file, the program checks for missing files (probably makes some suggestions) and let the user relink the files.
5. Fancy optional stuff #2: Let OpenShot have an option to pack a project (i.e. the project will be stored in a new, separate folder/archive with all project source files gathered therein).

[ggallozz (ggallozz-gmail)]

mustn't be a WHISHLIST !
it's a main usage issue !