INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Extreme memory usage (leak?) on large handheld sync  (Read 748 times)

BryanC

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2661
Extreme memory usage (leak?) on large handheld sync
« on: September 06, 2023, 02:00:24 pm »

I'm using the handheld sync tool to sync to an external 18TB hard drive. When MC is first scanning the drive ("Working..." in the device info area) I'm seeing some pretty extreme memory usage (this is after about 15 minutes of loading time, and climbing):



This repeatedly brought down my entire machine until the OOM killer took out MC. To workaround, I assigned 2TB of swap space on an extra SSD I wasn't using, but thought I should still bring it up here in case there is anything that can be done to reduce the memory usage of the handheld sync for large drives.

Logged

BryanC

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2661
Re: Extreme memory usage (leak?) on large handheld sync
« Reply #1 on: September 06, 2023, 02:12:33 pm »

Logged

BryanC

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2661
Re: Extreme memory usage (leak?) on large handheld sync
« Reply #2 on: September 06, 2023, 05:59:18 pm »

I think this was being caused by an illegal character. I believe it was due to a directory having "Florilegium (Medieval & Renaissance)" as the Album Artist (using the default layout). This raises an additional question, could all of these illegal characters be cleaned by default, I have implemented the following audio filename expression to workaround the issue and mimic the default settings, but it would be nice to know that using the default settings wouldn't cause issues to begin with.

Filename:
Code: [Select]
If(IsEmpty([Disc #],1),1,[Disc #])PadNumber([Track #],2) - RemoveCharacters(Clean([Artist] - [Name],3),.,0)
Directory:
Code: [Select]
Music/RemoveCharacters(Clean([Album Artist (auto)],3),.,0)/RemoveCharacters(Clean([Album],3),.,0)/
Logged

bob

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 13874
Re: Extreme memory usage (leak?) on large handheld sync
« Reply #3 on: September 06, 2023, 07:23:39 pm »

I think this was being caused by an illegal character. I believe it was due to a directory having "Florilegium (Medieval & Renaissance)" as the Album Artist (using the default layout). This raises an additional question, could all of these illegal characters be cleaned by default, I have implemented the following audio filename expression to workaround the issue and mimic the default settings, but it would be nice to know that using the default settings wouldn't cause issues to begin with.

Filename:
Code: [Select]
If(IsEmpty([Disc #],1),1,[Disc #])PadNumber([Track #],2) - RemoveCharacters(Clean([Artist] - [Name],3),.,0)
Directory:
Code: [Select]
Music/RemoveCharacters(Clean([Album Artist (auto)],3),.,0)/RemoveCharacters(Clean([Album],3),.,0)/
What character was "illegal" ?
Logged

BryanC

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2661
Re: Extreme memory usage (leak?) on large handheld sync
« Reply #4 on: September 06, 2023, 09:34:09 pm »

I'll try to reproduce next week when I get home, but it was one of the files in the attached playlist. Plenty of suspects, unfortunately. Quoted album names, long artist lists, parentheses, forward slashes, semi-colons, etc. Line 305 is especially odd with so many nested quotes.
Logged
Pages: [1]   Go Up