INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Maximum number of supported directories at one level  (Read 2984 times)

tlenthe

  • Recent member
  • *
  • Posts: 25
Maximum number of supported directories at one level
« on: December 08, 2018, 03:31:17 pm »

I recently consolidated all of my music files from multiple drives onto a single 8TB disk.  When I ran import on the new disk, I noticed that some of the last directory entries in the audio->files list were missing.  I did a little more investigation and it appears that there is a limit of ~512 directories at one level as that's about where the entries stopped.  All the content in the directories beyond the cut-off point are not present in the database for any searches, etc.

I can work around this by adding some additional "artificial" levels of directories, but as drives get larger, this might be a limitation that should be reconsidered.

Running Media Center 24.0.64 (64-bit) on Ubuntu 18.04

Logged

mwillems

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 5234
  • "Linux Merit Badge" Recipient
Re: Maximum number of supported directories at one level
« Reply #1 on: December 08, 2018, 07:25:32 pm »

I have about 3000 directories in one parent directory that MC scans, and it picks them all up (it does not stop at 512). 

Is what you're seeing repeatable (i.e. does it happen when you trigger a manual import)?  Any special info about the disk?  The filesystem, etc.?
Logged

tlenthe

  • Recent member
  • *
  • Posts: 25
Re: Maximum number of supported directories at one level
« Reply #2 on: December 09, 2018, 07:27:42 pm »

I tried manual import multiple times with no change.  After "nesting" folders it did get all the way to the last directory.  The directory is mounted from a network location using nfs.  I thought I checked the directory using system tools at the mount point, but I'm not 100% sure.   Is it possible this is a linux/ubuntu nfs artifact? 
Logged

bob

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 13875
Re: Maximum number of supported directories at one level
« Reply #3 on: December 10, 2018, 10:05:42 am »

I tried manual import multiple times with no change.  After "nesting" folders it did get all the way to the last directory.  The directory is mounted from a network location using nfs.  I thought I checked the directory using system tools at the mount point, but I'm not 100% sure.   Is it possible this is a linux/ubuntu nfs artifact?
I wonder if there is inode limit for dirs that's set in the NFS options somewhere?
Logged

mwillems

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 5234
  • "Linux Merit Badge" Recipient
Re: Maximum number of supported directories at one level
« Reply #4 on: December 10, 2018, 05:47:22 pm »

Is it possible this is a linux/ubuntu nfs artifact?

I'm not sure.  I don't have much experience with NFS; I can say I've never seen any similar problems using samba/cifs FWIW.  It might be worth testing with either a direct connection or an alternate network protocol to rule out NFS.
Logged

bob

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 13875
Re: Maximum number of supported directories at one level
« Reply #5 on: December 13, 2018, 04:13:31 pm »

I'm not sure.  I don't have much experience with NFS; I can say I've never seen any similar problems using samba/cifs FWIW.  It might be worth testing with either a direct connection or an alternate network protocol to rule out NFS.
I'd like to think that NFS would work better because there should be less filesystem attribute matching going on but there are a lot of people trying to make cifs work well with linux.
Logged

Manfred

  • Citizen of the Universe
  • *****
  • Posts: 1038
Re: Maximum number of supported directories at one level
« Reply #6 on: December 13, 2018, 04:36:21 pm »

If think that I had something similar in the past - I had one directory path with a file too long filename inside. Win10 had problems with that, it does't show the correct size of all folders. I used then a shorter filename.
Logged
WS (AMD Ryzen 7 5700G, 32 GB DDR4-3200, 8=2x2+4 TB SDD, LG 34UC98-W)-USB|ADI-2 DAC FS|Canton AM5 - File Server (i3-3.9 GHz, 16GB ECC DDR4-2400, 46 TB disk space) - Media Renderer (i3-3.8 GHz, 8GB DDR4-2133, GTX 960)-USB|Devialet D220 Pro|Audeze LCD 2|B&W 804S|LG 4K OLED )

tlenthe

  • Recent member
  • *
  • Posts: 25
Re: Maximum number of supported directories at one level
« Reply #7 on: January 25, 2019, 09:14:27 am »

I just wanted to close this out...  Whatever was happening resolved itself after rebooting the nfs server.  So, bottom line, MC24 wasn't the issue.

Ted
Logged

bob

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 13875
Re: Maximum number of supported directories at one level
« Reply #8 on: January 25, 2019, 04:15:04 pm »

I just wanted to close this out...  Whatever was happening resolved itself after rebooting the nfs server.  So, bottom line, MC24 wasn't the issue.

Ted
Thanks for reporting back.
Logged
Pages: [1]   Go Up