Topic: Handheld Sync: Huge file fails to convert/sync (Read 1779 times)

d_pert · « **on:** February 04, 2020, 12:37:05 pm »

Is there some upper limit to the file size which MC can convert from/to in Handheld Sync?

I've got a huge DXD ALAC track: 1.85 GB. (Yeah I know, it's ridiculous.)

MC can convert it just fine to MP3 CBR 320kbps using the normal Convert Format feature.

However, when MC tries to convert it to MP3 CBR 320kbps within the Handheld Sync process -- simply to another Windows folder on an internal NTFS HDD -- it always fails at 74%. MC briefly flashes "Error" in the Status column of the sync queue, and then then that's replaced by "In Queue".

I sync-convert zillions of files of all formats and sizes without trouble; only this one fails, and only in Handheld Sync.

The file is probably the largest single track file among my approx. 200,000 files.

Thank you.

RoderickGI · « **Reply #1 on:** February 04, 2020, 06:03:31 pm »

I suspect there is a timeout limit on each file as part of Handheld Sync, as that would be required when talking to an external device. So you are hitting the limit with that file. That suspicion is supported by your report that the transfer always stops at 74%.

Convert the file locally and only Sync the converted version.

If you turned on logging in MC, it would probably show the timeout issue.

d_pert · « **Reply #2 on:** February 04, 2020, 07:54:05 pm »

Thanks. Yeah: Doing a manual convert and just placing the file will take it out of the queue for good. Great idea.

I used the Search bar in Settings to look for Logging. No such keyword. Can you tell me how to turn it on? I'd love to look at it for a number of reasons.

RoderickGI · « **Reply #3 on:** February 04, 2020, 08:28:20 pm »

https://wiki.jriver.com/index.php/Logging

You can also just include the converted file in the MC Library, add it on its own to a separate Playlist, and then include that Playlist in the Handheld Sync Options. That way any real changes to the file would sync, and if you have it configured Number of Plays would sync back to MC, and so on. Using that method would keep all file syncing/coping to a device in the one process. Easier to manage.

d_pert · « **Reply #4 on:** February 06, 2020, 01:08:43 pm »

Maybe the timeout value could be made into an advance option/setting? “Handheld sync timeout”?

RoderickGI · « **Reply #5 on:** February 06, 2020, 02:25:02 pm »

It would probably be good to confirm the timeout diagnosis with a log before considering software changes. It is only a suspicion at the moment. I don't work for JRiver, and don't have access to code.

d_pert · « **Reply #6 on:** February 06, 2020, 10:38:47 pm »

I ran Logging while recreating the error. But the log file is vast. Should I post it here? Could you generously suggest any keywords to search for the error report? I tried, "error", and "timeout".

Update: I just realized I could attach it the Logging ZIP. Attached.

RoderickGI · « **Reply #7 on:** February 07, 2020, 10:02:58 pm »

What is the full path and filename for the problem file?
Does the filename happen to start be "H:\LIBRARY_HD_Stereo\Beethoven - Complete Symphonies (de Vriend) DXD\37 - Symphony No. 9 in D minor, Op. 125 'Choral'_ IV. Finale_ Presto.m4a", which is 141 characters?
What device is the Portable installation install on, and what format is the device formatted to?

I used Notepad++ to view the log, and searched for "Handheld", which showed a lot of records, no actual timeout, but the above file was removed late in the conversion process, PID 13948. I'm thinking maybe the device doesn't support file names over 128 characters, or similar. Or the portable installation itself doesn't support long file names. There is a setting for that, and there were some fixes in the past, but I don't know if they relate to the Portable install.

d_pert · « **Reply #8 on:** February 08, 2020, 09:39:38 am »

Hi RoderickGI:

Thanks for the in-depth look at this.

To answer your questions:

--The file I'm having trouble with is the one you ID'd.
--The portable installation is on an NTFS-formatted SSD.
--You didn't ask, but: The target 'handheld' device is a folder on an NTFS-formatted HDD.

Observation:

The character limit thing is a good thought. However, there are many other files successfully syncing in the same session which have substantially longer filenames/paths. I just added the 'Filename ~~(path)~~' column to the Handheld Sync list view and did a quick compare. So... ?

d_pert · « **Reply #9 on:** February 08, 2020, 10:17:23 am »

Update:

I tried all three Options > File Location > Conversion Cache settings -- in case it has to do with disk Cache or RAM max. size limits.

No change.

RoderickGI · « **Reply #10 on:** February 08, 2020, 03:51:43 pm »

The log entry that appears to stop the transfer is, or remove the problem file from the transfer list, I'm not sure which, is:
64104734: 13948: Database: CMediaDatabase::RemoveFileAbsolute: Filename: H:\LIBRARY_HD_Stereo\Beethoven - Complete Symphonies (de Vriend) DXD\37 - Symphony No. 9 in D minor, Op. 125 'Choral'_ IV. Finale_ Presto.m4a Mode: 0 Track: 0

The first number is a time code in milliseconds since logging was started. So logging has been running for 17.8 hours. You probably would have gotten a smaller log if you had Reset (cleared) the log file before testing. But it looks like the log captured the event, so no issue.

The second number is the Process ID, or PID. I followed that PID but couldn't see the real cause of the issue. But maybe it isn't logged. I'm not that great at reading these logs anyway.

As both your devices are NTFS, the file name length shouldn't be an issue. You have confirmed that now. The is a setting in Options to "Support extra long file names" which you could try, but I don't have it checked and haven't seen issues. Most past issues were fixed without that setting checked.

So basically I can't tell why that file isn't transferred. As it isn't name length, it could just be purely file size. Sorry. Can't help.

d_pert · « **Reply #11 on:** February 08, 2020, 04:15:24 pm »

Thanks for this.

I tried it again with the "Support extra long filenames (experimental)" feature turned ON:

No difference. Same error.

Because it's by far the biggest file I have, I'm persuaded to think it's just blowing some time/memory limit.

~~However, I can try to reacquire the file from source, just to see if it's a corrupt file. It plays fine all the way through, though.~~

Update: Wasn't thinking in the line above: normal conversion in Playing Now works fine, and converting with other tools (e.g., dBpoweramp) works fine. So it's not the file.

d_pert · « **Reply #12 on:** February 15, 2020, 05:43:56 pm »

Update:

You can right-click > Delete a queued item in the sync list before hitting Sync. That way it'll be excluded from the Sync operation. However, if you hit 'Recheck Sync', it'll come back.

d_pert · « **Reply #13 on:** February 16, 2020, 08:46:32 am »

Further update:

I just made some changes to a Smartlist which is one source driving the content list for for a Handheld Sync.

I let it use .DSF versions of tracks which I had previously written it to use the .M4A versions of (I keep both original .DSF and a parallel set of 88.2 kHz 'converts' of those).

Many more errors are being produced, of the kind described in my OP.

Not all DSFs fail, however. I compared the ones failing with ones not, and they're the larger ones by far.

So again -- assuming this is just a timeout or file size limit thing affecting Handheld Sync -- it would be GREAT to have a way to adjust these.

Can anyone please confirm/verify known time and/or file size limits affecting Handheld Sync?

Can these be made variable via a buried option please?

d_pert · « **Reply #14 on:** February 19, 2020, 09:58:33 am »

'Elegant' workaround until control of time/size threshold(s) is made an option:

I added a rule to disallow syncing files with sample rates higher than 192kHz. This blocks all DXD (huge) files. It also blocks all DSF files, but I keep a parallel set of PCM 88.2kHz converts of those.

Matt · « **Reply #15 on:** February 19, 2020, 10:18:41 am »

I just tried syncing a file that's a bit over 4 GB and saw that the progress information was overflowing.

So next build we'll have this:
Fixed: Uploading files larger than 4 GB to a handheld would make the progress corrupt.

d_pert · « **Reply #16 on:** February 19, 2020, 10:25:42 am »

Hooray! Thanks Matt!

Smartlists (re)editing ... here I come.

Matt · « **Reply #17 on:** February 19, 2020, 10:34:26 am »

I should note that I haven't stumbled onto any kind of timeout. It just slowly but surely works for me. Like I said, the progress information is all wrong, but I think it finishes eventually with the current build.

d_pert · « **Reply #18 on:** February 19, 2020, 10:45:18 am »

Interesting. I never just waited a while and then checked to see if the output file had actually been created. Although ... I did note that the Sync queue always re-queued the file which had "errorred"; so I assume the file had not actually been made.

d_pert · « **Reply #19 on:** February 19, 2020, 07:11:29 pm »

Matt: I got your dl confirmation IM, but cannot reply with IM:

"You have exceeded the limit of 3 personal messages per hour."

Yes, I am converting to .mp3 320kbps VBR "Extreme", no DSP.

Matt · « **Reply #20 on:** February 19, 2020, 07:19:48 pm »

Thanks! I tried at home and it failed. I'll look at why in the debugger tomorrow. Thanks again!

d_pert · « **Reply #21 on:** February 19, 2020, 07:38:42 pm »

That file might be the largest released single track file ever produced.

Matt · « **Reply #22 on:** February 20, 2020, 09:51:35 am »

You've found a file that's just bigger than our MP3 encoder accepts. WAV and APE work, but MP3 does not in CBR or VBR mode.

I've played a little and also sent the file on to others on the team, but not sure if you just found the limit or we'll be able to find a way around.

Thanks for your help.

d_pert · « **Reply #23 on:** February 20, 2020, 11:41:29 am »

The file converts to MP3 VBR 320kbps "Extreme" using Illustrate dBpoweramp -- which also uses the L.A.M.E. MP3 encoder.

Matt · « **Reply #24 on:** February 20, 2020, 12:47:23 pm »

Next build will address this problem. Here's the history:
Fixed: LAME MP3 encoding failed on files over roughly 15 minutes in length.

d_pert · « **Reply #25 on:** February 20, 2020, 01:35:38 pm »

Interesting. File play duration. Wouldn't have thought.

d_pert · « **Reply #26 on:** February 26, 2020, 04:30:15 pm »

I can confirm that with 26.0.29 and:

"4. Fixed: LAME MP3 encoding failed on files over roughly 15 minutes in length."

MC got through my "massive" DXD file using Handheld Sync encoding to MP3. Never was able to before.

Hooray and thanks, Matt!

JimH · « **Reply #27 on:** February 26, 2020, 04:39:31 pm »

To be clear, it was the encoder, not MC, that was failing.

mwillems · « **Reply #28 on:** February 29, 2020, 10:55:04 am »

Hey thanks for this! I had a 1.4G, 38-minute-long FLAC that would not convert to MP3 during handheld sync. It had been erroring out for a year or two but it was the only file that did that and I just assumed the file was defective in some way (although it played fine).

In the latest build it just converted and synced across, and I couldn't figure out why until I found this thread!

INTERACT FORUM

Author Topic: Handheld Sync: Huge file fails to convert/sync (Read 1779 times)