INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: "Real time" Library Server/Client Sync  (Read 5858 times)

cncb

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3105
"Real time" Library Server/Client Sync
« on: October 18, 2010, 09:03:22 am »

rpalmer68:  I'm also not sure I like the fact I have to sync the library or close MC to push the delete to the server.  I can undersand not wanting to push tagging changes to the server in real time, but I feel file operatons like this should get pushed to the server immediately just like TV subscription changes do.

Matt: We may do that eventually.


I would also like to request this happen in real time when files are added to the server library.  This would be useful when TV is being recorded so it will show up on a client without having to do a manual sync.  This also would be useful when auto-import folders pick up new files so they would be visible to a client right away (copying photos to an auto-import folder, for example).
Logged
-Craig    MO 4Media remote and player:  Android/TV/Auto | iOS | Windows 10/UWP

rpalmer68

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2639
Re: "Real time" Library Server/Client Sync
« Reply #1 on: October 18, 2010, 03:18:50 pm »

+1

I totally agree, having to manually sync on the clients for my TV recordings to appear is annoying, and in fact most of the family never remember and complain that the recording they are looking for isn't there and I have to do the sync for it to turn up!



Logged

ThoBar

  • Citizen of the Universe
  • *****
  • Posts: 992
  • Was confishy
Re: "Real time" Library Server/Client Sync
« Reply #2 on: October 18, 2010, 07:52:36 pm »

+1

I still dont sync remotely because of this, and it makes newly tagged/imported media a pain to find.
Logged

raldo

  • Citizen of the Universe
  • *****
  • Posts: 1102
Re: "Real time" Library Server/Client Sync
« Reply #3 on: October 19, 2010, 06:01:45 am »

family never remember and complain that the recording they are looking for isn't there and I have to do the sync for it to turn up!

+1

I've tried using server/client but found the whole setup cumbersome and I grew tired of intermittent behavior.

MC's client/server setup doesn't currently do what I think most users would think a client/server solution should do.
Logged

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: "Real time" Library Server/Client Sync
« Reply #4 on: October 19, 2010, 11:01:51 am »

I have to agree.  The current system still doesn't quite do it for me.  I run with my different copies of MC pointing directly to the library on the network drive (using the Read-Only access for subsequent connections).  This works because I have a limited number of "stations" of MC around the house (basically just the HTPC, the office, and my laptop), so it isn't a serious problem to manually manage them (though it really stinks when I have to run down to the basement to "reclaim" the read/write access for my HTPC).

We need a real client/server system, that allows multiple simultaneous "clients" to connect to the server, all with transparent read/write access (and some sort of conflict resolution system).  I don't think it needs to be the fanciest system on the planet, but the current system isn't quite there.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72439
  • Where did I put my teeth?
Re: "Real time" Library Server/Client Sync
« Reply #5 on: October 19, 2010, 11:11:37 am »

... and some sort of conflict resolution system ...
There's the rub.

Think about the food fight we've had over search suggestions the last week or so.  Multiply it by 100 and that's the direction you're proposing.

Why not a journaling file system on top of it?

We could do it.  I just don't think it would affect our sales.
Logged

cncb

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3105
Re: "Real time" Library Server/Client Sync
« Reply #6 on: October 19, 2010, 11:16:50 am »

If the updates were done in "real time" why would there be "conflicts"?  The changes would be pushed to all clients instantly so you would be working on the current database.
Logged
-Craig    MO 4Media remote and player:  Android/TV/Auto | iOS | Windows 10/UWP

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72439
  • Where did I put my teeth?
Re: "Real time" Library Server/Client Sync
« Reply #7 on: October 19, 2010, 11:26:17 am »

We could do it.  I just don't think it would affect our sales.
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42373
  • Shoes gone again!
Re: "Real time" Library Server/Client Sync
« Reply #8 on: October 19, 2010, 11:39:18 am »

We have said we would like to make syncing with the server more automatic.

If there are other specific changes you would like, please post your most important one or two.

The client / server model JRiver has chosen is "real" (glynor's words).  Clients have a full copy of the database and can use it at full speed or with an intermittent connection.  Changes are merged to and from a server, and any number of concurrent users can be supported.  This approach works a lot like SVN, a successful multi-user database system that I think is a work of art.

Logged
Matt Ashland, JRiver Media Center

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: "Real time" Library Server/Client Sync
« Reply #9 on: October 19, 2010, 12:17:24 pm »

The client / server model JRiver has chosen is "real" (glynor's words).  Clients have a full copy of the database and can use it at full speed or with an intermittent connection.  Changes are merged to and from a server, and any number of concurrent users can be supported.  This approach works a lot like SVN, a successful multi-user database system that I think is a work of art.

Fair enough.  It certainly is "real" as it is now.  I just need a more full-featured system to meet my needs.

Like I said... I don't think the current system is that far off from what I'd need.  The syncing does need to be much, much, much more automatic.  Sure, a fancy system with user rights, logins, and the whole thing COULD be useful, but that isn't required to make the existing system useful.  But the current implementation isn't quite there yet, mostly in the "transparent to the user" category.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: "Real time" Library Server/Client Sync
« Reply #10 on: October 19, 2010, 12:29:44 pm »

But, since you asked...

We have said we would like to make syncing with the server more automatic.

If there are other specific changes you would like, please post your most important one or two.

That's basically it.  I need it to work with all fields, and all file types, just like accessing a "shared" library does, but with all of the copies of MC on the network to have some version of read/write access.  I agree that the SVN "check in/out" model is probably the best one to go with, considering the existing infrastructure.  However, the check in/out process needs to be almost entirely automatic and asset-level granular rather than library-level granular.  So, for example:

I need to be able to do this:

1. Have one user (wife/children) watching a show on the HTPC (which involves "tagging" files with Number of Plays, Last Played, and Bookmarking, among other things).  Maybe they will do a few "light tagging" duties too.
2. Simultaneously have another user (me probably) on my laptop or in my office working with media files and tagging other things.  Changes to the library seamlessly sync to the HTPC without having to close and restart or anything like that (there can be a delay of some kind, that's fine).
3. If both users try to "tag" a particular file simultaneously (which would count when a file is playing back elsewhere), then a "read only" error message should appear, maybe showing which machine has the file "checked out", allowing you to continue but changes will be lost (so two machines can simultaneously play back one file), and maybe even giving you the option to "force check out" the asset.

If I can do that, it would go a LONG way towards making the system much more usable.  The last thing might be some sort of "server managment" system that can check to make sure the Library Server stays running on the serving machine, disconnect clients on a schedule to make backups and whatnot, upgrade to new versions automatically, etc.  These things may or may not be needed depending on the implementation.  So long as it can sit there rock-solid stable for a month, and make backups of the library, and everything.

We could do it.  I just don't think it would affect our sales.

I would pay for a "Pro" or "Server Edition" license.  I think others would as well.

Think about the food fight we've had over search suggestions the last week or so.  Multiply it by 100 and that's the direction you're proposing.

Tech enthusiasts often don't respond well to change, which is odd since they're enthusiasts in the most quickly-changing industry in history.  Personally, I was pretty enthusiastic over the suggestion engine changes.  With the changes in the latest build, I think it is about perfect.

But yes, changing the system dramatically would probably rile up the troops.  That's why I think the best solution would build on what you already have.
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

rpalmer68

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2639
Re: "Real time" Library Server/Client Sync
« Reply #11 on: October 19, 2010, 03:12:35 pm »

But the current implementation isn't quite there yet, mostly in the "transparent to the user" category.

I think the key for me at the moment is the "transparent to the user" statement.

For me the biggest real world issue is that my central HTPC records all my scheduled TV recordigs, but the kids watch these recordings on a different PC.

So if the LS clients could somehow automatically update when new files are added to the library, then my life would become a LOT easier.

Now given TV schedules are already sync'd back to the sever immediately (which had me confused for a while as nothing else does) I was just hoping that maybe physical file changes could be treated the same way.  (If deletes are too hard like this, then maybe at least addition)

We also have an MCC command to sync the library, but no option to do it silently.  As requested previously, if we could have a "silent" option for this command, then I could get each of my clients to sync overnight at different times to at least ensure all other changes, deletes (maybe) and  tagging etc get updated at least once a day.

Just these couple of things would mean the user experience for our household would be more pleasant, and there'd be less "Dad, where's my program" requests!

Don't get me wrong, I'd love to be able to RIP, import and manage cover art from a LS client but I can wait for this as long as my "users" have a better expeience.

Cheers
Richard
Logged

Mars

  • World Citizen
  • ***
  • Posts: 199
Re: "Real time" Library Server/Client Sync
« Reply #12 on: October 19, 2010, 03:29:50 pm »

Quote
it would go a LONG way towards making the system much more usable

I absolutely agree.

For me would also be a great value the ability of importing media from the client side.
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42373
  • Shoes gone again!
Re: "Real time" Library Server/Client Sync
« Reply #13 on: October 19, 2010, 03:58:28 pm »

For me would also be a great value the ability of importing media from the client side.

And would it then upload it to the server?
Logged
Matt Ashland, JRiver Media Center

Mars

  • World Citizen
  • ***
  • Posts: 199
Re: "Real time" Library Server/Client Sync
« Reply #14 on: October 19, 2010, 04:42:51 pm »

Quote
And would it then upload it to the server?

Yes, so the clients could also feed the library (the server) with new stuff.
Logged

raldo

  • Citizen of the Universe
  • *****
  • Posts: 1102
Re: "Real time" Library Server/Client Sync
« Reply #15 on: October 20, 2010, 01:52:47 am »

I just don't think it would affect our sales.

A well working client/server (implementing the core of the suggestions in this thread and other threads), would definitely set MC apart from the crowd. In one more way, that is.

There are endless possibilities in a well working distributed media system. Adding that extra computer (tablet?) in the kitchen wouldn't be an issue since MC seamlessly displays and plays your media.

I have to say that I've hesitated (slightly) adding an extra htpc in the basement. Most of this hesitation originates from "management overhead".
Logged

MrHaugen

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 3774
Re: "Real time" Library Server/Client Sync
« Reply #16 on: October 20, 2010, 06:32:19 am »

It would help a lot just to be able to set a timer for this merge process, would it not? If you could set a customized interval (each 15 min, 30 min, 1 hour, 2 hours, 6 hours, 12 hours, 24 hours etc.), this would feel almost like a always-up-to-date system. Depending on how you use it of course.
Logged
- I may not always believe what I'm saying

cncb

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3105
Re: "Real time" Library Server/Client Sync
« Reply #17 on: October 20, 2010, 10:54:14 am »

The more I think about this, I might be able to get the behavior I want by using separate (local) libraries for each client each auto-importing the same network folders.  Each local library should pick up new files and file changes pretty quickly and automatically.  Each client would also have its own saved Now Playing list, etc.  However, this would require MC to support saving playlists and smartlist definitions in external files and auto cleanup for missing/deleted files (if it doesn't already).  Any thoughts on this?
Logged
-Craig    MO 4Media remote and player:  Android/TV/Auto | iOS | Windows 10/UWP

gappie

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 4580
Re: "Real time" Library Server/Client Sync
« Reply #18 on: October 20, 2010, 11:06:02 am »

The more I think about this, I might be able to get the behavior I want by using separate (local) libraries for each client each auto-importing the same network folders.  Each local library should pick up new files and file changes pretty quickly and automatically.  Each client would also have its own saved Now Playing list, etc.  However, this would require MC to support saving playlists and smartlist definitions in external files and auto cleanup for missing/deleted files (if it doesn't already).  Any thoughts on this?
i do the same. i have problems though with video's. often the xml files are not made with a complain about files not there, and i think when an xml file is changed the autoimport on the other machine does not recognize it. but for audio and images it works great..

 :)
gab
Logged

cncb

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3105
Re: "Real time" Library Server/Client Sync
« Reply #19 on: October 20, 2010, 11:12:34 am »

i do the same. i have problems though with video's. often the xml files are not made with a complain about files not there, and i think when an xml file is changed the autoimport on the other machine does not recognize it. but for audio and images it works great..

How do you share Playlists?
Logged
-Craig    MO 4Media remote and player:  Android/TV/Auto | iOS | Windows 10/UWP

gappie

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 4580
Re: "Real time" Library Server/Client Sync
« Reply #20 on: October 20, 2010, 11:16:44 am »

How do you share Playlists?
im sorry.. should have added that.. i dont. guess you could export them to a folder in the shared location, and autoimport can pick them up then. but never tried that.

 :)
gab
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42373
  • Shoes gone again!
Re: "Real time" Library Server/Client Sync
« Reply #21 on: October 20, 2010, 01:21:07 pm »

I hope a build available in the next few weeks will include these changes:

NEW: 'Sync Changes with Library Server' no longer requires a reload of the client library when the server changes.
NEW: Library Server clients can automatically (and silently) stay synchronized with the server (Options > Media Network > Library Server Client > Automatically sync changes made on client back to server)
Changed: Bookmarks are shared (and synchronized) across Library Server and its clients.
Logged
Matt Ashland, JRiver Media Center

glynor

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 19608
Re: "Real time" Library Server/Client Sync
« Reply #22 on: October 20, 2010, 01:25:33 pm »

This is how this news makes me feel:
http://www.youtube.com/watch?v=fCGIeHp9GT0
Logged
"Some cultures are defined by their relationship to cheese."

Visit me on the Interweb Thingie: http://glynor.com/

nwboater

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1346
Re: "Real time" Library Server/Client Sync
« Reply #23 on: October 20, 2010, 02:09:10 pm »

It makes me feel just wonderful all over!!!

Thanks for pursuing this Matt.

Rod
Logged

rpalmer68

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2639
Re: "Real time" Library Server/Client Sync
« Reply #24 on: October 20, 2010, 03:07:11 pm »

Just call me Mr Happy!

Thanks Matt.

Richard
Logged

cncb

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3105
Re: "Real time" Library Server/Client Sync
« Reply #25 on: October 20, 2010, 03:32:37 pm »

This sounds excellent.  Thanks.
Logged
-Craig    MO 4Media remote and player:  Android/TV/Auto | iOS | Windows 10/UWP

Mars

  • World Citizen
  • ***
  • Posts: 199
Re: "Real time" Library Server/Client Sync
« Reply #26 on: October 20, 2010, 03:56:54 pm »

Great News!  ;)

And about the ability of importing media from the client side, Would its implementation be feasible in the future?

Thanks

Logged

park

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2358
  • I wish I had more to say!
Re: "Real time" Library Server/Client Sync
« Reply #27 on: October 20, 2010, 09:37:03 pm »

Great News!  ;)

And about the ability of importing media from the client side, Would its implementation be feasible in the future?

Thanks



Perhaps related to this request. How about application of cover art from a client? I manually grab all my video cover art from the net, and have to remote desktop into my server to do it at the moment.
Logged

MrHaugen

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 3774
Re: "Real time" Library Server/Client Sync
« Reply #28 on: October 21, 2010, 06:42:55 am »

Let's hope it's a bit more than a hope from Mats side :)

If this is sorted out, I can finally start using the library server. Have been waiting for this for years now. REALLY looking forward to it!
Logged
- I may not always believe what I'm saying

JustinChase

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3276
  • Getting older every day
Re: "Real time" Library Server/Client Sync
« Reply #29 on: October 21, 2010, 07:53:29 pm »

Very good to hear Matt!!

Thanks :)
Logged
pretend this is something funny

drmimosa

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 690
Re: "Real time" Library Server/Client Sync
« Reply #30 on: October 22, 2010, 07:17:18 am »

Quote
NEW: 'Sync Changes with Library Server' no longer requires a reload of the client library when the server changes.

Everytime I think "I wonder if Media Center does..." a forum discussion like this pops up. I think this feature would be a great addition to the Media Server.

JRiver Devteam...better than Nethack.
Logged
Pages: [1]   Go Up