Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Frans]

When I highlight a single album in JRiver and use the rename/move/copy function to Copy a single folder or a few folders, it works as expected: the folder and music contents appears on the external USB drive.

However, when I highlight my whole library (80k files) and Copy it the same way. JRiver ONLY copies folders and subfolders. No music files are copied at all. I am expecting all music files and all sidecar files to be copied (PDF, BMP, JPG, etc). I even tried this a few times in a row with the folder names already created - still no files being copied. The windows copy dialog comes up and shows 0 files being found and copied and promptly disappears.

How can I fix this? Is there copy file number limit?

I plan to eventually plan to move everything to this new drive but want to do this dry run.

More detail:
I want to move my library to another harddrive and delete all remaining empty folders and folders with a single jpg in them. Regular Explorer Copy/Move & Paste will just bring this legacy junk along.

Currently everything is read directly from a NAS, but I want to read directly from the attached USB 8TB drive, and back it up to the NAS via Acronis True Image. This allows me to create a second backup rather than trusting the NAS with everything.

[~OHM~]

are you using the most current update for MC24 ?

[Frans]

Yes, always the latest version. I do recall this being an issue before JR24, but I used Windows Explorer and did not worry about the empty / folder.jpg folders.

[swiv3d]

This does work - however copying very large numbers of files from a NAS to a USB drive is slow, the popup showing file movements will take a long time to actually show any actvity although if you look in the target directory you will see all of the folders which are empty and prepared for the transfer whenever it does get around to starting. I have just run a test copying 685GB of files from my NAS to a USB 3 drive - after about 15 minutes of calculating the time etc it now has started moving the files but tells me it will take 19 hours.

[fitbrit]

How soon after you start the process are you checking? That's a lot of files, and it will take MC a while to copy the stuff over, just like it would take Windows Explorer. Are you giving it enough time?
What happens if you choose 2 albums, or a hundred. Try 50 and wait a few mins to see if the files show up.

[Frans]

I let it run overnight and well into the next day, but JRiver indicated no activity when I checked.

It was effectively "done" doing anything after the folders had been created. The Windows File dialog initially showed 0 files found and calculating, but then it disappeared and never returned again. No more activity after it disappeared after about 5 minutes.

I should at least see some sort of progress report after a day, right?

[swiv3d]

The windows transfer popup shouldn't be disappearing - I suggest you select a few hundred files and transfer those just to check out your network. You might want to try just moving all of these files using windows explorer to see if that fails. This doesn't sound like an MC problem.

[Frans]

When I do a simple Explorer copy and paste it works - the the PC is now prepping all the files to be copied. With JRiver Copy it never gets past the Calculating 0% stage before disappearing. At this point it will have only copied the folders and it will not do anything else. I know that one or a few folders will work because I do it all the time when I import and move/rename things. I can try to do 1000 at a time, but that kind of defeats the purpose.

Right now, after about 10 minutes or so, Windows Explorer it has passed the calculation phase and has started physically copying files. The only difference is that I was asking JRiver to batch move my library folders and all their contents. It seems to be some sort of memory overflow issue possibly when large libraries are being copied.

 

[JimH]

Did you try updating the NAS firmware?  What is the NAS?

[Frans]

Synology 413J NAS, with the latest DSM Firmware. It's working fine through Windows Explorer which is copying now, and will take more than a day (as expected), but it is trudging through the 100,000 plus files (I have 85k music files which is why I want to cleanse this library).

[swiv3d]

I wonder what the limit is before MC ducks out of the transfer? I know 685 GB works ok.

EDIT: Works fine with 1.9TB but MC's memory load goes up to 1.2GB so perhaps a memory thing when the transfer is huge?

[Frans]

I too suspect it's a memory thing when the files get up there. 88k files, 5+ TB of music and files. Best I can do is try to move things 1 TB at a time.

[JimH]

I too suspect it's a memory thing when the files get up there. 88k files, 5+ TB of music and files. Best I can do is try to move things 1 TB at a time.

Very unlikely to be an MC problem.

[blgentry]

So the job you want done is to remove all folders within your music folder which have zero music files in them.  Empty folders or folders that just have .JPG files in them.

I can see why you would try to use the Rename, Move, and Copy files tool for this job.  Since it seems to be failing, I would consider using a different tool outside of MC.  My first instinct is to write a script to do it using Perl or some other scripting language.  But I realize that most MC users aren't skilled at scripting and that's probably not the right tool for most people.

I would also consider exporting a playlist of the entire library and using that to drive a copy program to do the copies externally.  Though that *probably* requires some scripting as well unless you can do something clever with a spreadsheet and perhaps a copying tool that can import a spreadsheet or a CSV type file.

Which leaves us where?  Maybe a third party utility for locating files?  Perhaps you can find one that will find folders that are empty and/or folders that only have JPGs in them.  That would probably get you where you want to be.

Your idea of using RM&C and doing it in chunks, will probably work great.  I would personally suggest doing it in smaller chunks than 1TB if you decide to go this route.  I would probably go extra painful and do it by album name or something like that, doing the As, then the Bs, then the Cs, etc.

You could certainly test it this way and see if MC is able to handle doing the copies when broken up this way.

It's an unusual and large task.  I hope you find a satisfactory way to get it done.

Brian.

[leezer3]

Stupid question-
Are you using the x32 or the x64 version of MC?

This is somewhere the x64 version might actually be of benefit.

Having said that though, I wouldn't be trying to copy at all.
Take a look at this StackOverflow post for some ideas:
https://superuser.com/questions/39674/recursively-delete-empty-directories-in-windows

[swiv3d]

Good point @leezer3, my system is 64 bit - don't know the OP's system setup. The robocopy solution would only be a partial fix  for him, as he says there are folders with just a .jpg  in them.
 @maid has a similiar problem
https://yabb.jriver.com/interact/index.php/topic,117390.0.html

[leezer3]

It's at this point I'd write a custom Bash script and run it on the server

Someone has already done something similar for some ideas:
https://gist.github.com/QuietMisdreavus/8304998d19b13b0085ee9db5c2b97eb0

Similar in Powershell:
https://superuser.com/questions/597802/how-to-batch-delete-folders-with-only-one-file-in-it

I would be *very* careful though of trying anything of this nature without backups. Far too much to go wrong for a few empty folders.

[Frans]

Win10 64-bit on this end, for what it's worth. Explorer is almost done copying everything.

Maybe, once it is all on the new drive I can try rename/move to a subfolder and test something on the same drive and not my main music folder. 

[swiv3d]

Good luck Frans hope it all works out as you want

[Frans]

Aside from not having enough space it works when done on the same drive. It's just pulling it off the NAS causes the folders-only copying bug.

[swiv3d]

Must be a synology thing then, my netgear NAS drive doesn't cause any problems in this respect. I would do a Move of half of the copied files - then delete those from the copy - then move the other half and you should achieve the outcome you wanted. Good Luck!

[Frans]

I looked into third party software, but there is none that is able to not copy empty folders or folders with only 1 file in them.

I need to start completely over because I ended up (after moving files over to the USB drive) in a state where I had Copied files using JRiver when I meant to Move them and ended up with trying to move files into a folder where they were present already. This was partially because I ran out of calculated space on the drive.

Whenever the error about replacing a file in the destination directory popped up the dialog box did not show the usual bulk copy "do this for all further errors" checkbox and wanted me to click Replace a thousand times before completing the task.

So starting over with a Windows Explorer (or alternative tool) copy from NAS to USB and then to use JRiver to Move everything into a music subfolder. This should give me a clean start. Hopefully when this is done I will have a best practices method for moving mass files off a Synology NAS.

[Frans]

I used FastCopy 3.54 to copy all files from my NAS. With verify on it took 24 hours. 91,599 files 12,056 folders.

Once on my USB Drive, I created a MUSIC subfolder and moved everything with the "Move All Files in Folder" checkmark on with just about all filetypes checked. Then I moved the files (30,000 per batch) only to find only FLAC files were Renamed and Moved - none of the PDF files JPGs that were in the folders, these were all left behind. This has always been an issue while importing net new folders - I've had to manually copy the album art files because they are always left behind.

I blamed this on my unmonitored staging area. This is an issue now because I am trying to move everything.

Attached is a screen of the settings where I have clicked the move everything including, well everything. Why does this not work?

[swiv3d]

Hi Frans
I ran a simple Rename test, set up a library called test library and imported a single folder with 27 hires flac files, a pdf booklet and a large  jpg file. Selected the audio files with F6 Rename with the Move unimported folder contents checked and moved them to a new folder on the same drive, They went from folder Test to folder Test2 - all of the files including the pdf and the jpg ended up where they were supposed to go and the delete empty folder popup appeared. Can't understand why you are losing the unimported contents from your folders when they are moved.

[swiv3d]

Difficult to see why the unmodified items in the folders disappear since on the same drive all that  MC is doing really is changing the directory structure - the question arises were the unimported folder contents actually there before the F6 F,M&C process was invoked. Copy won't move the folder contents but rename definitely does.

[Frans]

Not sure what that means about of the files were actually there. Is there a way to know whether JRiver included them?  Below is a picture of my usual import settings.

Also there is an example of the files left behind after the rename/move function. I don't know if this has ever worked for me. Not sure what else I need to do. I do recall that when I import the folder it actually acknowledges the presence of these extra files.

[swiv3d]

I think the problem might be that you have imported all of the files from every folder so MC will only move the audio files you have selected but leaves all of the other imported files (jpgs, pdfs, etc) where they are because they are within your library and you didn't choose to move those ones.

[blgentry]

If it's a matter of all of your "extra" files being imported, then you should see these imported files in another view, like a Data view or a Pictures view.

For tasks like this, I suggest that you build a utility view that shows EVERYTHING regardless of Media Type.  An everything view can be very helpful for figuring out weird media problems.

Brian.

[swiv3d]

I would suggest that you move the audio files back where they came from. Then clear the library and do a fresh import with just audio, and iso files ticked. Then do the move to the music folder, perhaps trying out with just a couple of albums first to see that it does work as you want.

[Frans]

Not sure I follow the last "move the to the music folder" step. Do you mean Importing only Audio into JRiver from a fresh copy, and then doing the Rename/Move with the "all files" checked?

[swiv3d]

If you have a fresh copy of the folders with all of the files intact, then you can import the audio files - but do not import all of the other file types - then do the rename move with all files selected. Try one folder first to check that it is working correctly - mine does but that isn't a guarantee that your's will work to.

[RoderickGI]

Just be aware that having the "All files" setting checked means that all files in the sub-directory where the audio file is located will be moved, not just the extra files associated with the audio file.

So basically that setting is only useful if every Album, Single, or other media file resides in its own sub-directory, and you don't have sub-directories that contain unrelated media files. Consider what would happen if you had a sub-directory containing an Album, and then moved just one Track to another location. I'll leave that as an exercise for you. 

PS: While I think I understand why you have imported all these files, have you looked into the "View Extras" functionality in MC? I don't think there is a Wiki topic on it, but if you search the Release Notes for "View Extras" you will find note on all the file types supported. Most are in MC22 I think, but check the later version notes.

Also search the forum for "View Extras". Here is the announcement of the functionality. https://yabb.jriver.com/interact/index.php?topic=106819.0

[Frans]

Hello all,

Just wanted to tie-out the topic. I re-copied everything from my NAS to USB and imported ONLY the music, ISO, and CUE files. I usually copy every filetype JRiver support. This is where the issue started I suspect.

Then used Rename/Move and everything went as hoped. At this point it left a ton of folder.jpg files behind (probably because these images are embedded in the individual files). So far so good I think?

I went down from 91k files to 88k files and down 2000 folders that were left over from from operations ver the years. I will test this new library to see if everything is right and no cover art is lost, and eventually will make this my main library (and then backup all these new files to my NAS.) I did find about 50 files that had normal file sizes but were corrupt, JRiver put these in an Unidentified folder so I can clean up later.

It would be great if the Copy feature didn't fail with large libraries, but FastCopy did great and bypasses Explorer (which on its own would have worked well, but I had more control with FastCopy. Somewhere between Explorer and JRiver there is a buffer overflow when copying large libraries.

In the end, my goal was to eliminate all the empty folders or folders where the much had been moved out, but redundant files were left behind. In this respect I have finally succeeded! About 45GB worth...

Thanks for all the help!