Topic: Auto Import Problem (Read 5725 times)

OmarL · « **on:** July 03, 2014, 06:45:33 am »

Not exactly sure how to articulate this problem but it's frustrating. About once or twice a week auto import will reorder my recently added albums so rather than showing the true order of recently added MC19 will display several rows (100 albums or so) of old material then a portion of recently added. In addition to this error it is consistently very slow. I do have a lot of material....wait....I just went to check how much exactly and it appears files have been removed or are not recognized! darn....seems this is worse than I thought. Ok...I thought there was at least 3.2TB but it's now showing 3TB. I have my library on a Synology NAS in one folder, "main library". It's organized as main library>artist>album.

Any help on this? I've searched through the forum and cannot find any matches.

JimH · « **Reply #1 on:** July 03, 2014, 07:28:58 am »

Could a drive letter be moving around? Sometimes it's E:, sometimes F:, for example? Windows can do that when you add and remove drives.

Make sure your Synology firmware is up to date.

OmarL · « **Reply #2 on:** July 03, 2014, 07:32:46 am »

Firmware is current. I actually don't have a drive letter so maybe this is causing some of the issue? The strange thing to me is it used to work. I'd have an issue but maybe every 3-6 months. Only recently has it become frequent.

OmarL · « **Reply #3 on:** July 03, 2014, 07:55:56 am »

I just mapped a drive for the library, now "Z". Not sure if I have done this correctly as it's not something I have done often. I did this on my server (CAPS Zuma) so now the library (Drive Z) shows up under "Computer" rather than "Network". I reconfigured auto import removing the old target and adding only Z (under the Computer tree). It is scanning now so I will monitor and add some new files once it's done to see how auto import is working.

Did I set this up correctly?!

adamt · « **Reply #4 on:** July 03, 2014, 08:44:59 am »

Quote from: OmarL on July 03, 2014, 07:55:56 am

Did I set this up correctly?!

It sounds good to me. Let us know how it works out.

OmarL · « **Reply #5 on:** July 03, 2014, 10:07:30 am »

Brief delay because I changed the library location in auto import but forgot to change it in the main library location so everything was showing duplicate. Had to clear it all and start over so now it's rebuilding from scratch. Good thing already is it's correcting another issue I had with cover art not showing up in a lot of albums. Now they are all showing up and doing so quickly. As soon as it's completed the scan I'll try adding some new album to test auto import. Taking some time, it's over 3TB.....

OmarL · « **Reply #6 on:** July 03, 2014, 01:19:40 pm »

Something is still not right, scan came to a grinding halt after 2.3TB, about 1TB shy of completion. Closed MC19 and reopened but that didn't correct it so I'll reboot and see if that helps.

OmarL · « **Reply #7 on:** July 04, 2014, 06:47:57 am »

Last night it finally finished loading my library although I had to go in and manually start auto import 4 times. This morning I woke up and 1TB was not showing, now it is loading again. I'm starting to think some of this is related to network issues too. I replaced my wireless router last week and have been getting drop outs since. That is another pain, BTW! It seems this is somewhat common with this router so I am actually going to try another one today. Maybe it fixes the library issue too? Maybe partially but I had it before I replaced the router, just not as pronounced. The drive was not mapped though so possibly doing that along with correcting the dropouts could be a solution.

OmarL · « **Reply #8 on:** July 04, 2014, 02:07:45 pm »

This is frustrating, any help would be greatly appreciated.

I did try another router which has stopped the intermittent interruptions on my network. I thought this corrected the library issues too. Scan was completed and my full library showed. I added some albums and after about 15 minutes auto import did work and they appeared. About 3 hours later I noticed now about 1/10 of my library was gone again. Also more albums that I added did not show up this time. I'm at a loss for what to do next.

JimH · « **Reply #9 on:** July 04, 2014, 02:28:18 pm »

Try setting up a local library as a test. That might tell you whether the NAS is a factor.

OmarL · « **Reply #10 on:** July 04, 2014, 03:04:40 pm »

I can try it but it seems to be more of an issue that is impacting a large library. A small library, which is all I'd be able to put on the Zuma, may not challenge things much.

JimH · « **Reply #11 on:** July 04, 2014, 03:30:05 pm »

It might help eliminate the Synology as a source of the problem. It has had trouble in the past.

http://yabb.jriver.com/interact/index.php?topic=77683.msg529084#msg529084

and

http://yabb.jriver.com/interact/index.php?topic=69707.0

JimH · « **Reply #12 on:** July 04, 2014, 04:46:01 pm »

Take a look at the Import options. You can tell MC to protect files on removable drives. If your drives spin down and are unavailable from time to time, it might account for the problem you're seeing.

6233638 · « **Reply #13 on:** July 04, 2014, 05:19:55 pm »

Disable the "fix broken links" option for auto-import until you sort out your networking problems.

OmarL · « **Reply #14 on:** July 06, 2014, 06:34:09 am »

Here's my update...

I think I've narrowed this down to having some relationship with either the server/Zuma or NAS dropping off the network, this seems to happen during periods if inactivity. Nothing is set to sleep and the connections are wired so not sure why that is happening at all but overnight seems to be when the library loses some files. I use remote desktop with my laptop to the Zuma to control MC19 and periodically that will disconnect too, this wireless. It only lasts a few seconds and automatic re-trys reconnect pretty quick. All of these drops in the network occur when devices appear to be getting a strong signal, another baffling factor for me.

Other than that, my library remained pretty stable yesterday. I did have it playing most of the time so not much inactivity. I also added albums and they were picked up properly with auto import.

Re "fix broken links" setting, it was set to "yes". I've disabled it to see if that will help.

I very much appreciate the input.

OmarL · « **Reply #15 on:** July 06, 2014, 11:44:27 am »

Something else I noticed, when the dropout occurs there seems to also be a dropout from the Zuma and MC19 because normally if I pause or stop in mid playlist or album when i return and hit play again it picks up on the track it left off. Now if I return after one of these drops it starts at the beginning of the library showing, as if the player had just been turned on.

glynor · « **Reply #16 on:** July 06, 2014, 05:27:02 pm »

In that case, turn off Fix Broken Links in the Auto-Import settings.

MC tries to tell if a source file is on a Network Volume that might be just disconnected, but their ability to do this reliably is pretty limited by the information the Windows API provides. Turning off Fix Broken Links will stop Auto-Import from EVER removing missing files from the Library, under any circumstances. (Once broken, they'll stay broken.)

And try to fix your network issues. Sounds like it could be:

1. Bad cabling.
2. Something broken (needing new firmware maybe) in your gateway or router.
3. Something borked in your network stack on one of the systems (bad Network Device driver, perhaps, or just crappy laptop or onboard hardware).

If it is isolated to Wireless (it sounded like it wasn't) then it might be interference.

glynor · « **Reply #17 on:** July 06, 2014, 05:32:59 pm »

Also, if you're using mapped drive letters on your client machines, you'll probably have better luck importing the files into MC using their UNC paths directly instead. Eg:

\\servername\share\path\to\the\files.flac

Rather than:

M:\path\to\the\files.flac

MC always treats UNC paths as network volumes, and expects them to disconnect from time to time, so the Fix Broken Links feature to protect network files works more reliably. It also has the benefit of not forcing you to manually map matching drive letters on all of your computers, so it is less of a pain too.

Auto-Import can monitor network shares in UNC format the same way it monitors all other filesystem locations.

OmarL · « **Reply #18 on:** July 17, 2014, 07:33:03 am »

Sorry for the delayed reply, we had a family emergency that drew most of my time and energy

Good news though, I think I have this/these issues resolved and it was mainly an issue with my LAN setup. I can't pinpoint exactly what corrected it, possibly a combination of things. I pretty much put in a new network...new router, modem and drivers. I think the main fix came from the driver in the Zuma' network adapter (I thought this was current but some digging revealed a later version from Intel).

This new driver update installed a ton of adjustment features, one of which was speed and duplex. For some reason my the network adapter was not connecting properly with the router when set to auto, changing it to full duplex/1000 solidified the connection. I use an Acoustic Revive LAN filter at the LAN input of the Zuma (can't say enough good things about that filter...another thread, though) and for some reason setting the LAN adapter to full duplex it would not see the cable with that filter in, I think because when set to full duplex it will only see other full duplex connections and that filter is not (a guess). So ....my challenge was then to somehow get the whole thing to work in "auto" so I could continue to use my filter. After a few restarts, for whatever reason it eventually did. That was yesterday and for the last 24 hours there has been no dropped LAN connections, my filter is in and MC19 seems to be running like a top.

The one remaining issue is after periods of idle there is a slight lag in getting MC19 to respond, possible from trying to communicate with the NAS. I do still have the NAS set where the internal hard disk and external SATA discs hibernate after 20 minutes inactive. I'm a little nervous to change this as I don't want to reduce the life of those drives. My sense is that if I do remove that setting the lag will disappear...so maybe I should.

The last test will be to change the auto import settings back to how I had them, the fix broken links setting mainly which I will try.

Just wanted to wrap this up and post my outcomes. Thanks to all who contributed and I hope this helps someone down the road who may experience similar issues.

glynor · « **Reply #19 on:** July 17, 2014, 08:32:19 am »

Parking drives does not... repeat, does NOT... reduce wear on the drive mechanism. In fact, it could be argued that in most circumstances, it increases wear.

The drive is a motor. It spins. Once it spins up, angular momentum provides the vast majority of the energy required to keep it spinning (you're basically just fighting friction and the slightest impact from gravity).

Getting it up to ~5400rpm (or whatever), and breaking it to slow it down, is where you have the vast majority of the wear. Yes, bearings wear down over time when in use, but they're typically much more robust than other components, and parking/respinning hurts those too.

Parking the drives is all about:

* Saving power.
* Protecting the drive from damage from movement (drops, etc, in mobile devices).
* Reducing total system vibration (not typically very substantial, if you take basic steps to isolate the drives, unless you have a large array in a case).

OmarL · « **Reply #20 on:** July 17, 2014, 09:04:35 am »

glynor that is great information to know, thanks for the input. I'll go ahead and change that setting to never hibernate.

RoderickGI · « **Reply #21 on:** July 17, 2014, 10:19:35 pm »

Quote from: OmarL on July 17, 2014, 07:33:03 am

I think the main fix came from the driver in the Zuma' network adapter (I thought this was current but some digging revealed a later version from Intel).

Bingo.

Had I seen this thread earlier I would have pointed immediately to the Zuma's network adapter driver. I have seen a lot of network problems with the sort of symptoms you described. Getting the latest Intel driver has almost invariably fixed the problem, especially when a third party (motherboard or network card manufacturer) driver was in use initially.

It is also common in my experience to require several reboots of a PC before a new network driver settles down and works reliably. Well, at least two. Usually this is because files were in use requiring a reboot, and then after the first reboot the driver wants to make more changes, and requires another reboot. Behaviour depends on whether you download and install a complete (large) driver package, or a driver installer package that then wants to download and install more components. Sometimes it needs a reboot to complete negotiations with other network components for speed of connection.

Now that you have the Zuma adapter set to negotiate LAN connections automatically, you may wish to check what speed your Zuma is connecting to the router. You can check that using the router interface, or as I prefer go to the "Control Panel \ Network and Internet \ Network and Sharing Center \ Connections" and click the "Local Area Connection", and probably in the Intel driver dialogues. The real speed of the connection will be listed. It should be 1Gbps on any modern network adapter.

However, as you have an Acoustic Revive LAN filter on that cable, I suspect you will find that it running at 100Mbps. I tried to find specifications for the Acoustic Revive LAN filter but could not find any. What I did find was one person who said that it only supported 100Mbps, plus someone who reported it only had 5 of the 8 wires connected in the Ethernet RJ45 connector. Check yours. If it only has five pins connected to wires in the connector, then it only supports 100Mbps. Based on he fact that the Acoustic Revive LAN filter couldn't be seen when you set the Zuma Ethernet adapter to 1Gbps, I am going to assume that it does only support 100Mbps. (It wasn't the full duplex that stopped this connection, it was the speed the filter supports.)

What I suspect is that you have had network issues for some time, resulting in lots of dropped Ethernet packets and hence, resending of packets and a subsequent reduction in the real speed of your Ethernet connection. As the cable is connecting your Synology NAS to your Zuma, via the router I assume, I guess it could effect JRiver playback of the files by causing JRiver to wait for data, though this shouldn't happen because the data would be buffered at the Ethernet receiving end. If the network connection was bad enough though, the buffer could empty, and that would definitely cause audible problems with playback. The fact that JRiver was unable to find files on your NAS while running automatic import would suggest that you had very significant network problems.

If you had the network configured to "Auto" speed connection it would have been running at 1Gbps by default. When you installed the Acoustic Revive LAN filter into this data path, the network speed was renegotiated down to 100Mbps and the reliability of the network connection would have improved dramatically. That may have been why the Acoustic Revive LAN filter had such a positive effect on the listening experience. (Yes, I am a Acoustic Revive LAN filter sceptic. Digital data is digital data. Noise on an Ethernet cable doesn't effect audio playback. However, a poorly performing network certainly does.)

Now that you have fixed, or improved, the network reliability at the Zuma end you may wish to check that you are using good quality CAT6 or CAT7 Ethernet cables, and double check that the Synology NAS has no outstanding Ethernet adapter issues, and make sure that new router has the latest firmware with no issues. You may even wish to find a utility to check if you are getting dropped packets at the Zuma end of the connection. Or you could just run the "Ping" command in a Windows CMD window from the Zuma to the NAS.

If all is good you may wish to try removing the Acoustic Revive LAN filter and test the listening experience with the network running at 1Gbps, then separately at 100Mbps. Your router, set to auto speed configuration, will negotiate the connection speed to whatever the Zuma network adapter is set to, when the Acoustic Revive LAN filter isn't inline.

While even a 24 bit 192kHz stereo LPCM sound track isn't going to saturate a 100Mbps Ethernet connection, other activities such as copying files to and from the NAS, importing new music, audio analysis and any other library maintenance functions would benefit from a 1Gbps connection, so it might be worth trying to live without the Acoustic Revive LAN filter. Unless, of course, the Acoustic Revive LAN filter does work at 1Gbps, and it makes the music sound better for you.

I would also convert back to using UNC paths. Much more reliable and easier to maintain. Once the network is working properly.

OmarL · « **Reply #22 on:** July 18, 2014, 07:31:36 am »

First, a short update. I remove the hibernate setting on my NAS and as expected the slight lag corrected. It appears all is running proper now.

Roderick thanks much for all the thoughts, tips, info etc. A lot to digest! Like you, my first thought was to make sure the LAN adapter driver was current however my lack of experience/knowledge reared it's ugly head right from the start. I assumed that simply asking Windows to check for a current driver on the web was doing the job. Well...not so. It took some digging to find the Intel driver that did eventually help get things right. Computers humble me, it never fails. Your comments explaining why reboots are often necessary to get the driver up to speed make good sense. I bet that is exactly what happened in this case.

You were also correct in how the speeds were impacted by the AR filter. In place they auto adjust to 100mbs, without 1Gbs. That line does only one thing which is to service the Zuma so the speed is not really a factor....unless it drops completely but that now seems to be corrected. I do need to verify that in the logs which is were very helpful in diagnosing some of this mess but from performance it sure seems corrected.

As for the filter, I can only say I highly recommend trying it. You may find benefit, you may not but in my system the results are exceptional. I don't think that has anything to do with the network issues I've been addressing. It's service is more about RF/EMI and noise on the line in general. In fact it's so effective I am strongly considering the LAN cable Acoustic Revive sells because clearly protecting that signal transmission as much as possible seems to be very important to the end result. That doesn't surprise me too much given the experiences I've had experimenting with USB cables, AES3 cables and the like. Anyways.....I think the impact of filters like that AR (all sorts of similar devices used in medical environments) are measurable too if that is your thing. I know a lot of guys need to "see it", personally I could care less as long as it sounds good:)

RoderickGI · « **Reply #23 on:** July 18, 2014, 08:32:45 pm »

Yeah I'm not going to debate the benefits of the Acoustic Revive LAN filter. If it works for you, great. It is quite an expensive bit of kit though.

A CAT7 cable should be equivalent to the LAN cable Acoustic Revive sells as it is shielded as well. Have a look at the specs of both. A CAT7 fly lead may be considerably cheaper than the AR product, and more readily available. A properly implemented Ethernet port isolates the Ethernet cable from EMI effects of the connected device, or interference. USB on the other hand is grounded at each end, and can share power between devices, hence interference.

I'm lucky in that I can send all my audio over HDMI at present, so no need for Ethernet in the mix. That may change.

What I was suggesting was that you do a sound test now that you have your network issues "fixed", with and without the Acoustic Revive LAN filter. That would be a more reasonable test than your previous tests where the network was failing without the filter, but working okay with it, at a lower speed. You may well still see a marked improvement with the AR cable, which would be great. You may see less of an improvement because the plain Ethernet connection is working better. That is okay as well. At least it means that you are getting the best at each stage of the media transport.

INTERACT FORUM

Author Topic: Auto Import Problem (Read 5725 times)