INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Auto import in the background and Synology NAS  (Read 12971 times)

psam

  • World Citizen
  • ***
  • Posts: 153
Auto import in the background and Synology NAS
« on: March 12, 2013, 10:05:35 am »

It has been repeatedly posted that Auto import in the background does not work as it should on Synology Servers.

I ' ve taken up the issue with their technical staff for some time now,  and this is their last reply :

**********
Dear Panagiotis,

Thanks for your reply.

Unfortunately we do not support filesystemwatcher components so that
wouldn't work for JRiver as using the way to detects file changed on
DiskStation.
However, we will forward your inquiry to our Product Management team for
further consideration of inclusion, and they will do more research about
this feature.

BR,
Antoine Yang

***************
The guy even logged in on my 413j and performed some teste, but as you can see he reported back that currently they don't support the "file system events" functionality of Microsoft.

So, if any other JRiver users out there are having the same problem , or are thinking of buying a Synology NAS to use with their HTPC, I guess you should start sending them emails requesting them to include this functionality in the next DS version.

I don't know if the JRiver people could spare some time and communicate with Synology (I know you have don it for other products in the past) and explain what they need to do.

Until then, manual Auto Import every now and then is the only solution I'm afraid.
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42323
  • Shoes gone again!
Re: Auto import in the background and Synology NAS
« Reply #1 on: March 12, 2013, 12:34:47 pm »

If a watched folder fails to support watch events, we will manually scan it every two hours.  It's not perfect, but better than nothing.

You'll see 'Failed to watch folder' in the log file if there are folders like this (Help > Logging).
Logged
Matt Ashland, JRiver Media Center

rayooo

  • World Citizen
  • ***
  • Posts: 171
Re: Auto import in the background and Synology NAS
« Reply #2 on: March 12, 2013, 01:46:07 pm »

I guess I must be confused, as I have a Synology 1812+ and MC auto-import works fine.
I never have to do any manual importing.

I'd like to understand what problem I have that I don't think I have?  ::) ;D
Logged
MC-27-28> Meitner MA 1V2> CJ-GatV2> CJ ART 300s> Magnepan 20.7

psam

  • World Citizen
  • ***
  • Posts: 153
Re: Auto import in the background and Synology NAS
« Reply #3 on: March 12, 2013, 02:47:20 pm »

I guess I must be confused, as I have a Synology 1812+ and MC auto-import works fine.
I never have to do any manual importing.

I'd like to understand what problem I have that I don't think I have?  ::) ;D
Do you mean that as soon as you FTP a new album somewhere in your watched folder subfolders, it gets imported to your JRiver library?
Logged

rayooo

  • World Citizen
  • ***
  • Posts: 171
Re: Auto import in the background and Synology NAS
« Reply #4 on: March 12, 2013, 03:08:43 pm »

yea...

so just to be sure..  I've got a dedicated MC computer that is used exclusively for audio playback. Theater view, no mouse, no KB.
This dedicated MC computer is pointed to and watching a specific folder on a Synology NAS where music media is stored.

The dedicated MC computer is set to run auto import in the background. The watched folder is of course the music folder on the NAS.

Just yesterday, I downloaded 3 new albums from HD Tracks ( on a separate desktop computer )
Once dowloaded, the 3 Albums files, were copied from the desktop computer over to the Synology NAS music folder.

Once copied, I can go into the other room and begin to play the new albums, as typically within a minute or less, they are "auto-imported" to the dedicated MC computer.

I've done it like this now for over a year, so that's why I always "think" that auto-import works just fine, I do however expect that what I'm describing and thinking is in fact NOT what others are speaking of when stating that Synology NAS does not support background import.  

Let me know what I'm missing/overlooking!  Thanks!

Logged
MC-27-28> Meitner MA 1V2> CJ-GatV2> CJ ART 300s> Magnepan 20.7

psam

  • World Citizen
  • ***
  • Posts: 153
Re: Auto import in the background and Synology NAS
« Reply #5 on: March 13, 2013, 02:17:40 pm »

yea...

so just to be sure..  I've got a dedicated MC computer that is used exclusively for audio playback. Theater view, no mouse, no KB.
This dedicated MC computer is pointed to and watching a specific folder on a Synology NAS where music media is stored.
...


Could you elaborate on your setup/settings please?
I mean, even though Synology replied to me in writing that their software does not support the function responsible to trigger an auto import, I trust that your setup somehow overcomes this limitation.....

What OS are you using? How do you attach the network location?

Thanx
Logged

rayooo

  • World Citizen
  • ***
  • Posts: 171
Re: Auto import in the background and Synology NAS
« Reply #6 on: March 13, 2013, 08:26:59 pm »

Here is setup info....

Win7-64.

I use direct address to NAS locations. (not Mapped drives, I can never remember what the proper name for this is)
in my case the media location as well as auto-import watched folder is:  \\DISKSTATION\Music\HiRezAudio\
Virtually all features in MC not pertaining to audio are disabled.
All computers are wired Gig-E, standard network, HP Procurve 1810G-24 and a smaller Procurve at the Diskstations.
(I use two 1812+, one as primary storage, the 2nd as a backup to the first)

I was running a very old firmware in the Synologies, I did recently update, although the audio-import worked prior, and worked with MC-17 as well.
I continue to be at a loss.
Logged
MC-27-28> Meitner MA 1V2> CJ-GatV2> CJ ART 300s> Magnepan 20.7

zxsix

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1753
Re: Auto import in the background and Synology NAS
« Reply #7 on: March 13, 2013, 10:42:33 pm »


I use direct address to NAS locations. (not Mapped drives, I can never remember what the proper name for this is)
in my case the media location as well as auto-import watched folder is:  \\DISKSTATION\Music\HiRezAudio\

"UNC path" (universal naming convention).

I also use a Synology NAS using UNC path.
My difference is that I have a C:\import folder on my local hard drive on the main computer.  New songs are copied to there.  Auto-import watches that folder.  I prefer separation in that if the files are on the NAS, I know I have properly tagged those files.  Then I use the rename/move feature in MC after tagging to move the files to the NAS.
Any files waiting in c:\incoming I know I still need to finish tagging.
Just a workflow thing for me.

Can't tell you right now how quickly a new file introduced on the NAS would show up since I don't do that.  I do understand what Synology's rep means though.  Since the NAS runs a linux operating system, both the operating system and the disk formats are completely foreign to windows.
On a windows readable file system (fat32 or ntfs) there is a 'notification' function when a file is created or modified.  This 'notification' is used primarily by backup software to know that the file needs backed up again.  Windows can't detect that happening on the NAS because a) it's a network resource not a local resource, and b) not a compatible file system.
If MC does in fact pick them up, then it's because MC is being proactive about looking for changes (Jim mentioned it will scavenge every couple of hours at the latest), but it's not going to be doing it from the file change notification from the O/S.
Logged

bivan

  • Recent member
  • *
  • Posts: 20
Re: Auto import in the background and Synology NAS
« Reply #8 on: March 14, 2013, 03:30:19 am »

Hello,
I tried to dig some more into this issue and here are my findings.
I have identified a few scenarios regarding media files on NAS devices and how they are picked up by the OS hosting MC (Windows). If Windows OS is "aware" of the presence of these files, so will be JRiver MC (it has been explained several times how that works.)

1. Files are copied/modified/deleted onto the NAS device into one of the watched paths using Windows File Service (CIFS/Samba) from the computer hosting MC. Since it is the Windows hosting MC that creates these files, it is aware of them and MC picks them up immediately.

2. Files are copied/modified/deleted onto the NAS device from another host than the one hosting MC, through the use of CIFS/Samba. Samba server on the NAS device will be "aware" of the new files into its watched directories (shares) and triggers a CHANGE NOTIFY to its other clients. That means the Windows client hosting MC will be notified that new files are created/modified/deleted (etc) and the changes are picked up immediately by MC. I have replicated this and I can confirm that it does work (at least with Synology.) This is the reason why some users (eg. rayooo) report the auto-import feature as working (they copy the files on the NAS through Samba).

3. Files are copied/modified/deleted onto the NAS through any other means but CIFS/Samba (eg.: FTP, HTTP, SSH, torrent download, local creation, etc.)
The Samba server will not be aware of these changes and so, it will not trigger a change notify. It would be very nice if the linux kernel on the NAS would support "inotify" and that Samba server would be compiled to use such a feature. This feature means that Samba can poll the kernel for changes to the filesystems through the inotify() interface. From what I have read over the Internet, it does not. The configuration option for Samba regarding this is: kernel change notify=yes. It is a global configuration option. Unfortunatelly, for Synology, it does nothing. I assume support for this feature was not implemented because of performance issues.

How to get around this:

- manual refresh of the directory issued from Windows (a simple access to the directory containing new/modified files would suffice)
- manual restart of Samba on the NAS. For synology: /usr/syno/etc/rc.d/S80samba.sh restart (command line access is required).
- automated restart of Samba. There is a workaround for "detecting" the change of files in the filesystem. It is a software called inotify-tools and can be installed on synology. It can monitor (using inotifywait command) even recursively a directory and issue a message whenever a filesystem event is detected.
Then, through the use of a script, you can call a samba restart. Again, command line access is a must. More of this here: http://forum.synology.com/enu/viewtopic.php?f=37&t=56439.
Still, it is not a very good solution, since files change quite often in a large share (eg. creation of thumbnails by MC). That would lead to quite often restarts of the daemon affecting the normal usage.
- ask the nice developers of MC to implement a button/hotkey command/app command that would trigger a "global" refresh. The way it is done now is through (from memory): File - Import - Run Auto-Import Now. But this requires the use of at least a mouse and using the normal view; not so cool for HTPC's.
The performance impact (at least for me) is not an issue, because my library is quite small and it would finish fast. :)

Regards,








 


Logged

rayooo

  • World Citizen
  • ***
  • Posts: 171
Re: Auto import in the background and Synology NAS
« Reply #9 on: March 14, 2013, 06:20:17 am »

YES! that's it  ;D  UNC UNC UNC UNC... (me trying to remember the TLA..  :D )  Thank you  zxsix !


'and bivan, thank you very much for that explanation! it's starting to make sense now.. One of those things I've wondered about for ages.  :)
 
Logged
MC-27-28> Meitner MA 1V2> CJ-GatV2> CJ ART 300s> Magnepan 20.7

raediant

  • Recent member
  • *
  • Posts: 7
Re: Auto import in the background and Synology NAS
« Reply #10 on: April 28, 2013, 01:21:48 pm »

Hi.
This is the closest thread I have found to my topic enquiry. I use MC mainly for high quality audio.
Background.
My first HTPC used Win7 and MC17. It had a small SSD for the OS and a large HDD with all my music on it. Everything worked fine and MC imported all the files OK and built and maintained the library.
Now I've built my second HTPC using Win8 and MC18. Again there's a small SSD for the OS but no other storage on board. I decided to store all my music on a separate large physical hard drive attached to my main Win8 PC which does not have MC installed. (Later I plan to move all this music onto a Synology NAS together with my pictures and videos which is how I came upon this thread: very useful. Thanks). All my media on the main PC are stored on drive D and I've redirected Win8 so that My Pictures, My Music and My Videos are on drive D. I add/modify my media on the main PC. ie I rip CDs, download hi-res music, add and manage photos (Photoshop) etc onto drive D on the main PC. The HTPC and main PC are connected to my network by wired gigabit Ethernet and the HTPC acts as a music and Blu-ray (ArcSoft) "player". I don't plan to add/manage media using the HTPC.
Problem/s.
My confusion/concern is around importing/synching my music. As my HTPC running MC does not have any storage for media I expect it to go to the dedicated drive on my main PC over the network to access My Music or My Photos (and later on the NAS). By the way, on the main PC I've mapped My Music, My Pictures, My Videos as network drives M, P and V as I read somewhere else that would be a good idea to help MC pick up those from the network. After installing MC18 I stopped the auto import for the moment while I sort out what to do. Would import actually try import the media files to my HTPC? Where do they go? There's no room on my HTPC to import them. Or does import simply build a "library directory" so to speak on the HTPC pointing to the main PC drive D My Music My Pictures and My Videos (or main PC mapped drives M, P or V) where the media is? Since I'm managing my media on the main Win8 PC drive D, from what I've read on this thread MC on the Win8 HTPC should be able to auto-synch files changes etc. as they're both using Windows. Right?
Is my strategy of using my powerful main PC to manage my media (now on drive D, later on NAS) and using my HTPC with MC to build libraries and play the media a good one? Will it work OK? Are there any other points you feel I should consider?
This proposed combination is all a bit new to me so I'd appreciate any help/guidance. Thanks.
Logged

rayooo

  • World Citizen
  • ***
  • Posts: 171
Re: Auto import in the background and Synology NAS
« Reply #11 on: April 28, 2013, 06:13:44 pm »

Here is how I handle things, not exactly your needs.. but close

I do use a central Synology NAS for all media storage.  Several computers, with MC installed on each, point to this central NAS media storage share.
(could just as easily be a PC with a large shared "D" drive)

In my specific case, each individual PC with MC on it, maintains it's own library. This has it's limitations as opposed to having a "shared" library that all the players work from, but for me, the individual PCs, each with library, work fine.

The primary two MC players used are:
1. A play only, theater view only, (no mouse, no KB)
This player is for serious listening only, a dedicated appliance (SSD boot drive, Win7, MC) this PC does nothing but run MC in my listening room.
About the only thing I do metadata wise is change rating on individual tracks via the remote control.

2. This PC is used for ripping, downloading, primary metadata.. and playing music (desktop computer)

Here is a workflow example, and, an example of the beauty of this setup.
I added a couple new Test CD rips (FLAC) yesterday using of course PC#2 (PC #1 doesn't even have a CD/DVD Drive)
The two CDs were ripped, stored on the synology, I added basic metadata, but forgot to add any genre info.
Today, I'm in the listening room, and I notice I have two CDs with Unassigned Genres. (Smartlist)
When convenient, I go out to PC#2 and add the proper Genres to the two recordings.
Point being, the new genera info is updated only on the local PC#2 library.

I then walk back into the listening room, where PC #1 has been playing along while I made the changes.
I view the two previously generaless recordings, both ALREADY have the genre info.
This means that within no more than a minute or so, PC#1 while playing merrily along, detected a metadata change within the media storage and updated it's own internal library!  I think this is pretty slick.  ;D







Logged
MC-27-28> Meitner MA 1V2> CJ-GatV2> CJ ART 300s> Magnepan 20.7

thurston

  • World Citizen
  • ***
  • Posts: 160
Re: Auto import in the background and Synology NAS
« Reply #12 on: April 29, 2013, 08:22:16 am »

Marked for later.  Thinking about coming back to MC and using with my synology and theater view.
Logged

normd

  • Recent member
  • *
  • Posts: 31
Re: Auto import in the background and Synology NAS
« Reply #13 on: June 02, 2021, 07:55:54 pm »

I have a Synology NAS for my music files. If I use my MC Mac to download files directly from HDtracks to the NAS, auto import works. If I download ZIP files from another service onto my main Mac, unzip them and then copy all files to the NAS, the music does not auto import. Running import manually on the parent folder works every time on these files. Why the difference?
Logged

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2565
Re: Auto import in the background and Synology NAS
« Reply #14 on: June 03, 2021, 03:20:53 am »

As the first post says, Synology doesn't support FileSystemWatcher.

When MC runs it asks the OS (OSX): tell me about any modification on this folder - \\nas\share\music (or whatever). The OS will then request the NAS to warn whenever something changes on that folder. The problem is the the Synology Samba/NFS implementation apparently doesn't support these notifications, so it doesn't warn OSX when something happens. So MC doesn't hear about it.

When you change the folder on your 'MC Mac', OSX doesn't need a notification - it is the one doing it! So it warns MC that something changed on that folder. When you do it on a different Mac however, your MC Mac receives no notification so it can't warn MC.

The alternative is to have MC scan the disk to detect changes. That's what you do when you run the Import process manually or on a schedule.
Logged
Pages: [1]   Go Up