Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[mwillems]

Could you make a note that pressing upload when connected remotely to another jriver instance should not look for the path where the file is on the remote jriver instance? That's never going to work.
I never had this working on any machine yet either. On Windows I got well that. I would have mounted the directory to that path. But how do I mount to /data/music on Windows? ^^

Cross platform file paths are a lingering issue that's not resolved.  The current system works in homogenous environments (i.e. all windows machines using UNC file paths to a NAS, or all linux machines with a NAS share mounted to the same directory), but there's no way to get *nix style paths and UNC or windows style paths to play nicely in a mixed windows/*nix client-server environment with JRiver right now.  This has implications outside of cloudplay too (for example, the "play local file if available" check box won't work correctly on Linux clients of a windows server and vice versa). 

Something like a customizable "root-filepath" variable could at least partially solve the problem; A fix for this issue has been on my wishlist since MC for Linux launched, but I recognized it was previously a niche issue.  The path issue looks a little more front-and-center here, so it might be worth sorting out the filepaths issue for good as part of the fix here.

[JimH]

I agree.  It's time we found a solution.

[shortie]

I’ll try this when I get home tonight and report back

I tried this and got logged in to cloudplay in Panel but couldn’t find a way to upload a playlist there. Maybe that was implied in the post that said it wasn’t working yet on Linux but i thought I’d post anyway.

[max096]

Cross platform file paths are a lingering issue that's not resolved.  The current system works in homogenous environments (i.e. all windows machines using UNC file paths to a NAS, or all linux machines with a NAS share mounted to the same directory), but there's no way to get *nix style paths and UNC or windows style paths to play nicely in a mixed windows/*nix client-server environment with JRiver right now.  This has implications outside of cloudplay too (for example, the "play local file if available" check box won't work correctly on Linux clients of a windows server and vice versa). 

Something like a customizable "root-filepath" variable could at least partially solve the problem; A fix for this issue has been on my wishlist since MC for Linux launched, but I recognized it was previously a niche issue.  The path issue looks a little more front-and-center here, so it might be worth sorting out the filepaths issue for good as part of the fix here.

I dont think filenpath compatibility is a big problem. Other than for backups (if it does not work there, I dont know, didnt try to backup on Linux and restore on Windows)

You should never retrieve file paths that are true on a remote system and try to use them locally. Either get the file (instead of the path from the remote jriver) or make the jriver on the other end upload it for you.

[mwillems]

I dont think filenpath compatibility is a big problem. Other than for backups (if it does not work there, I dont know, didnt try to backup on Linux and restore on Windows)

It does break backups, it also prevents some media server options from working correctly (as noted above).  It's also a limiting factor (IMO) in allowing clients to make certain changes that are currently "server-only." 

You should never retrieve file paths that are true on a remote system and try to use them locally. Either get the file (instead of the path from the remote jriver) or make the jriver on the other end upload it for you.

In the server-client playback context, allowing clients to play "local" files if available dramatically improves the client experience when it works (seeking in videos, for example, works much better with direct file access than when relying on the server to serve the file, etc.). 

You're right that they don't need to fix file paths globally to fix the cloudplay issue, but fixing the filepath issue would fix this issue and a handful of other issues too if you see my point.

[max096]

In the server-client playback context, allowing clients to play "local" files if available dramatically improves the client experience when it works (seeking in videos, for example, works much better with direct file access than when relying on the server to serve the file, etc.).

Which is how it is currently. A network drive is not local either. It working well is proove that local files arent necessary and you could match or surpass say smb (for your specific purpose) without requiering users to puzzle together paths by mounting things in a way that matches the remote.

Say your remote jriver is windows and has its music on C://Music. You cant mount that in that way on another machine, even if its Windows too.
Its just not always possible to do that. And that all assumes people have a network share setup they have access to.

[BryanC]

Which is how it is currently. A network drive is not local either. It working well is proove that local files arent necessary and you could match or surpass say smb (for your specific purpose) without requiering users to puzzle together paths by mounting things in a way that matches the remote.

Say your remote jriver is windows and has its music on C://Music. You cant mount that in that way on another machine, even if its Windows too.
Its just not always possible to do that. And that all assumes people have a network share setup they have access to.

I don't think I'm following your logic here. mwillems is describing a setup in which there are duplicate copies of the media files, one on the server instance and one on the client. This is useful for, say, laptops where you may not have unmetered access to your library server on the go. You can save considerable bandwidth by playing the local file. I believe what you are referencing (and I may be wrong) is a single NAS share mounted on both the client and the server. These are different use cases.

The easiest solution from a theoretical standpoint is to just strip the OS-dependent prefix and perform some translation on the path separators (which I believe is already done in MC, albeit in an ugly way with escaping which is exposed in the handheld sync path settings).

[max096]

I don't think I'm following your logic here. mwillems is describing a setup in which there are duplicate copies of the media files, one on the server instance and one on the client. This is useful for, say, laptops where you may not have unmetered access to your library server on the go. You can save considerable bandwidth by playing the local file. I believe what you are referencing (and I may be wrong) is a single NAS share mounted on both the client and the server. These are different use cases.

The easiest solution from a theoretical standpoint is to just strip the OS-dependent prefix and perform some translation on the path separators (which I believe is already done in MC, albeit in an ugly way with escaping which is exposed in the handheld sync path settings).

Basically, yes youve got a NAS and you connect to it from everywhere else. Local cashing or syncing features are all also useful. But thats not really what this was about.

On my nas my music is in /mnt/raid/music lets say that I bind that into my docker container to /data/music. So inside the docker container /data/music/song.flac would be a valid path that exists. Now Im on my windows PC and connect to jriver inside the docker container, trying to upload a playlist witch contains song.flac. Now it will tell me /data/music/song.flac does not exist on the Windows. Even if you cache or sync it it would never be under /data/music/song.flac on the Windows PC unless I also mount it there.

Does that make sence now?

It may also just be a weird error message that was not really planned. For cloud sync that might very well be a case of "Not quite implemented yet". An error message like "Could not find /data/music/song.flac on 192.168.1.xx" would make more sence. Or just "Could not find song.flac" then. But if you where actually trying to find it at "/data/music/song.flac" on my Windows PC i´m 100% going to get the same error message once this works on Linux as /data does not exist on any of my Linux desktop installs.

[bob]

We just did a bit of noodling on this concept.

What if the database filepaths have separators and "drive" placeholder values that are filesystem agnostic (how they are stored is yet to be determined)
Then there are path mappings stored independently in a path settings file to tell each OS where that "drive" starts.

For examples of the drive placeholder values, in windows C: is mapped to the first drive whereas in Linux and Mac that might become /
For a network path a UNC path \\ becomes /media/foo/bar in linux or /Volumes/foo/bar on Mac.

We modify our existing file functions to test for equivalency not including the drive placeholder values.

One could then simple move a Library from Windows/Mac/Linux and then just update the path mappings on the destination machine.

This could apply to stream playlists as well I think.

Thoughts??

[Hendrik]

An automatic solution that tries to automatically split drives out is probably not going to be flexible enough, between mapped network drives on Windows, or arbitary filesystem mount points on Linux (or on Windows even, albeit more rarely used). How do you know that /a and /b are different drives? I suppose you could read the mount table, but that sounds like a nightmare to figure out.

Personally, I would go a simple but tad bit more manual way. Let the library continue to simply store a path, say the path on your primary media system, ie. your server, and on clients you can define "mappings" where you tell MC how to translate the path to the current system (ie. a simple string replacement at the beginning of the string, first rule to match wins with a defined order)

That way you also have zero risk of screwing up the library itself, and only have to make proper mappings. Safety of the library is key there.

[bob]

An automatic solution that tries to automatically split drives out is probably not going to be flexible enough, between mapped network drives on Windows, or arbitary filesystem mount points on Linux (or on Windows even, albeit more rarely used). How do you know that /a and /b are different drives? I suppose you could read the mount table, but that sounds like a nightmare to figure out.

Personally, I would go a simple but tad bit more manual way. Let the library continue to simply store a path, say the path on your primary media system, ie. your server, and on clients you can define "mappings" where you tell MC how to translate the path to the current system (ie. a simple string replacement at the beginning of the string, first rule to match wins with a defined order)

That way you also have zero risk of screwing up the library itself, and only have to make proper mappings. Safety of the library is key there.

Agreed, it probably wasn't clear, but that's the same approach I was trying to outline.
Matt was thinking we might be able to find a character that's illegal in all of the filesystems and use that for a path separator in the database. Thoughts?

[max096]

On my part I would quite like the idea of instead of trying to make absolute file paths platform independent. Don't. And instead

Add a "Library Part" entity that specifies the root path of the directory platform dependent and also gets a library id of let's say 1. When you put songs into that folder the song would be added with a relative path to wherever that folder is.

For path seperators a similar thing to the JVM seems to be true according to our overlords at stackoverflow / is save. https://stackoverflow.com/questions/122455/handling-file-paths-cross-platform
Never use backslash for the relative paths, it breaks everything, BUT windows. However, according to this guy on stackoverflow Windows does translate / into \ automatically. Previously, I thought this might have been specific to the JVM, but seems like it actually is more universal than that. But there are still platform dependent characters that are invalid on some systems and valid on others. So those are the one's witch could still make problems.

As far as the client goes since the music file has an id that is unique now. It later would be unique only within the library part. So you would need both the library part id and the song id. Then you can identify the song. Where the song is, is the business of the jriver playing the server role. If you cache the file locally, it does not need to be human readable. Hence something like cache_dir/1/5/song.flac would be fine. Giving it it's own directory makes it so you can possibly save both the real file and a converted mp3 should that ever be needed. When you import/sync an entier library with your notebook. You would need to supply it paths to map it to. It does not make sence to me to take the same paths, because ~/Music might be used for something else than it is on the remote PC. It does not seem like a very good idea to just "automatically" put files where you guess they should go without user intervention. Unless it's a cache directory that is in a jriver specific directory or one specified by a user who would run out of space otherwise.

When you do a backup you throw away the platform dependent root folder path entierly and only store the id. When you restore the backup the paths would need to be remapped. Witch might be additional ui steps. But it works and allows restructuring files when restoring a backup (witch I think would often be the case, you reinstall your PC, you might want to clean up a little bit).
On second though, you could still store the root path and make a check if the folder is valid and in fact the same. You (could) make that step optionally skippable as in "Run auto restore". Then a "Auto restore could not restore library 3 it possibly moved, where is it?"-type of response, meanwhile 1 and 2 succeeded. Now you gotta only map 1 folder instead of 3. If you where doing manual restore you'd be manning all of them. If you are switching operating system from windows to linux. You would also be mapping all of them.

---

Also, by the way if you say you don't want to do this in 25. I don't really expect you to in 25 quite like that (but you might have ideas that would be more compatible with your current solution of handling things). But come 26. You could change how folders are handled and until then find a solution that really solves the problem once and for all. I'd rather wait for it a year than have it somehow fixed with constraints you might have because of not wanting to break compatibility within the 25 release, unless you can come up with a way that truly works. I would say most solutions to this would likely break if you go down a version, because you won't have code in place that translates the path as it would later do.

[Hendrik]

Matt was thinking we might be able to find a character that's illegal in all of the filesystems and use that for a path separator in the database. Thoughts?

I just wouldn't put it into the database at all - ie. dont change how filenames are stored, deal with it at runtime. That way you have zero chances of messing up the database (accidentally by the user, or with a bug).
We shouldn't overengineer this. The majority of cases will remain simple filenames, and the library should remain consistent with that, for simplicity and safety reasons.

Making huge changes to how filenames are stored in the library is a very risky endavour - and quite simply not required to cover the client/server requirements.

[mattkhan]

I don't have a strong view on how it is done, the client side mappings seem sensible enough though it would be nice to make life easy for those who do use consistent paths on different OS's, for example the ability to define a default in the library for each OS (that is the not the server OS).  For me this would mean I could set mappings once on the server (like w:/ -> /media/music ) and all my linux clients would automagically work.

The other thing would be making it visible somewhere when this is happening so clients can debug when things are not working as expected.

[RoderickGI]

Please don't forget the use case where there is only one copy of media files, but the Client has direct OS access to those files using the same path as the Server, when the setting "Options > Media Network > Client Options (when connected to a library Server) > Play local file if one that matches Library Server file is found" is checked.

For example, in a Windows only environment, the Server has all media on the M:\ drive, with a suitable path. The Client has a mapped M:\ drive that points to a share on the server. Then when the Client plays a file, it finds the file directly using its mapped M:\ drive, and plays it directly. This improves seeking, as mentioned above, but also allow Blu-ray menus to be used on a Client, which is not possible without this capability. With this use case, files are accessed directly rather than streamed by the MC Server.

There was a long thread discussing possible solutions to the OS-agnostic path issue some time back, if anyone wants to search for it.


My other concern with some of the discussion above is that [filename], [filename (path)], and [filename (name)] must continue to work correctly. I think this would work best if the Library stored a value that was interpreted during runtime.

How about using a nomenclature like this?

:::Music::Artist::Album::File.ext

Where
::: means the base path, and would resolve to something like "M:\" in Windows and "/mnt/raid/" in Linux/OSX.
:: means the OS specific path separator


In the raw form, [filename,0] would contain ":::Music::Artist::Album::File.ext", but when used programatically, [filename] or [filename,1] would resolve to M:\Music\Artist\Album\File.ext in Windows, and something like "/mnt/raid/music/Artist/Album/File.ext" in Linux/OSX.

If you don't use a solution like that, where an interpretable value is stored in the Library, then you need more complexity to cater for situations where a user has a Linux Server and Windows Clients, as well as a Windows Server with Linux/OSX Client, or mixed Clients with either Server OS.

Just my 2 cents for now. I think I spent a dollar or two in the last thread on this subject.


...Oh, wait. The above wouldn't cater for my situation where I have media on multiple drives, each with there own drive letter, so there is no one "Base Path". Hmmm, more thought required. Not today.

[max096]

How about using a nomenclature like this?

:::Music::Artist::Album::File.ext

Where
::: means the base path, and would resolve to something like "M:\" in Windows and "/mnt/raid/" in Linux/OSX.
:: means the OS specific path separator


In the raw form, [filename,0] would contain ":::Music::Artist::Album::File.ext", but when used programatically, [filename] or [filename,1] would resolve to M:\Music\Artist\Album\File.ext in Windows, and something like "/mnt/raid/music/Artist/Album/File.ext" in Linux/OSX.

C:\\mnt\raid\music/artist/file.ext

Is something Windows could work with so I'd just use / as a path seperator. I'm not sure if windows explorer would let you go there (for open this file in windows explorer) if you gave it that path to open. Quite possible it still does still do that, but I´d have to try. Obviously you can´t manually input / into Windows explorer. But opening file streams programmatically should work.

[BryanC]

...Oh, wait. The above wouldn't cater for my situation where I have media on multiple drives, each with there own drive letter, so there is no one "Base Path". Hmmm, more thought required. Not today.

It would be OK as long as the client doesn't have multiple drives. If the server has multiple drives it shouldn't matter (if both drives have the same folder structure) since you'd be stripping the drive letter regardless. If the client has multiple drives then you would have to determine on which drive the folder exists. This would add some overhead. The other option is to require users to pool drives on the client at the filesystem level.

[mattkhan]

if this change is implemented, will it be sufficient to allow a windows client to play BDs from a linux server library via local disk access or is disk access a necessary but not sufficient condition for this feature?