INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Smartlist duplicated thousands of times  (Read 1271 times)

merman-corrode-portage

  • Recent member
  • *
  • Posts: 31
Smartlist duplicated thousands of times
« on: July 05, 2021, 10:29:15 pm »

I made a smartlist last week, which duplicated a couple of times after I changed the title from lowercase to a capitalised name. I left it over the weekend and came back to work to find it had duplicated more than 10,000 times. In addition I can't connect to my library and the MC server appears to be unresponsive. I did a library restore from backup which only worked for a couple of minutes.

The only hint in the logs is a repeating entry saying:

Code: [Select]
rm: cannot remove '/config/.jriver/Media Center 28/Temp/Delta': Directory not empty
and another repeating warning saying:
Code: [Select]
(mediacenter28:929): dconf-WARNING **: 13:11:03.252: failed to commit changes to dconf: Failed to execute child process ?dbus-launch? (No such file or directory)

Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72438
  • Where did I put my teeth?
Re: Smartlist duplicated thousands of times
« Reply #1 on: July 05, 2021, 10:43:05 pm »

Stuck key?
Logged

merman-corrode-portage

  • Recent member
  • *
  • Posts: 31
Re: Smartlist duplicated thousands of times
« Reply #2 on: July 05, 2021, 10:47:36 pm »

The smartlist was made from a client using the library remotely, the server is a headless machine. The client which created the list originally was shut down over the weekend.
Logged

merman-corrode-portage

  • Recent member
  • *
  • Posts: 31
Re: Smartlist duplicated thousands of times
« Reply #3 on: July 05, 2021, 11:52:41 pm »

I restored from an older backup and things seem to be stable now, however I'm curious to know what happened if anyone can shed some light on the matter.
Logged

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2616
Re: Smartlist duplicated thousands of times
« Reply #4 on: July 06, 2021, 03:20:25 am »

I believe that when you make changes using a Client, it periodically creates a 'Delta' file with the changes that need to be sent to the server for replication (same for changes made on the Server and sent to the Clients). In this case it seems that the same delta file was applied over and over, and each time MC failed to delete the file after applying the changes so it reprocessed it on the next loop.

Check the permissions on that folder, perhaps MC can write but can't delete files there. (EDIT: Nevermind, in Linux there's no separate Delete permission)

The Dbus error might be related or not (not sure if MC uses it - it's possible that the Dbus error is triggering the Delta issue, I don't know). I think the message means dbus-launch is missing, you can try to install that package. For Ubuntu, I've read somewhere that you need this:
sudo service xrdp stop
sudo apt-get install dbus-x11
sudo service xrdp start
Logged

mwillems

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 5234
  • "Linux Merit Badge" Recipient
Re: Smartlist duplicated thousands of times
« Reply #5 on: July 06, 2021, 08:25:28 am »

I'm with zybex, definitely check the permissions on the files in "/config/.jriver/Media Center 28/Temp/Delta" and on the directory itself.  It would also be interesting to know what the files in that directory were named.

Out of curiosity is the client machine also a linux machine or a windows machine?  Are there network file shares in the mix and if so what kind?  I ask because you mentioned changing the case of the smart list name, and I have sometimes seen strange behavior when changing case of filenames in a cross platform context because window is case insensitive and Linux is case sensitive.  Some of the weirdness I've seen isn't necessarily something MC could do anything about (i.e. the issue was at the network share level so MC can't fix it), but I've also seen some odd behavior in MC too that I couldn't reproduce. 

I'm not sure if smartlists are written out to files or only live in the database, but the case change might be a clue.  If my hunch is correct, you might be able to reproduce the weird behavior by creating a smartlist on the same client, manually syncing changes to the server, and then changing its case, and syncing again.
Logged

merman-corrode-portage

  • Recent member
  • *
  • Posts: 31
Re: Smartlist duplicated thousands of times
« Reply #6 on: July 06, 2021, 08:26:24 pm »

To be clear, I'm using the shiomax/jrivermc28 docker container, so I'm pretty sure all the dependancies, such as dbus-x11 are included, besides I noticed that error only appears when I'm using vnc to access the headless app so it's probably unrelated.

The files are all on the same machine as the jriver container with paths mapped to the container.

There are several client machines, however the particular one which made the edits to the smartlists is a Mac.

Code: [Select]
# ls -la /config/.jriver/Media\ Center\ 28/Temp/Delta/
total 56
drwxr-xr-x 1 app app   492 Jul  6 13:53  .
drwxr-xr-x 1 app app 13426 Jul  7 11:15  ..
-rw-r--r-- 1 app app   574 Jul  6 13:53  browser.jmd
-rw-r--r-- 1 app app    38 Jul  6 13:53 'field (bookmark).jmd'
-rw-r--r-- 1 app app    41 Jul  6 13:53 'field (date last opened).jmd'
-rw-r--r-- 1 app app   162 Jul  6 13:53 'field (date tagged).jmd'
-rw-r--r-- 1 app app    34 Jul  6 13:53 'field (last skipped).jmd'
-rw-r--r-- 1 app app  2225 Jul  6 13:53 'field (library merge info).jmd'
-rw-r--r-- 1 app app    30 Jul  6 13:53 'field (skip count).jmd'
-rw-r--r-- 1 app app    38 Jul  6 13:53 'field (zone last opened).jmd'
-rw-r--r-- 1 app app  1485 Jul  6 13:53  mediafiles.jmd
-rw-r--r-- 1 app app    16 Jul  6 13:53  platform.jmd
-rw-r--r-- 1 app app  5228 Jul  6 13:53  playlistx.jmd
-rw-r--r-- 1 app app    16 Jul  6 13:53  removable.jmd
-rw-r--r-- 1 app app   283 Jul  6 13:53  user.jmd

I performed the steps to attempt to reproduce the error but it seems to have synced all changes correctly.

EDIT: I noticed that the container default environment variable for GROUP_ID is set to 1000 where if I run "id me" on the host machine I get "id me
uid=1000(me) gid=100(users) groups=100(users)" so I added an -e flag to the container to modify GROUP_ID to 100. I'm not sure how to reproduce the error, but this could possibly have something to do with not being able to remove the Delta directory.
https://hub.docker.com/r/shiomax/jrivermc28

EDIT2: Prior to changing the environment variable for GROUP_ID, the "Delta" directory was still in my "Temp" directory.
Code: [Select]
root@me-server:~# ls -la /mnt/user/appdata/jriver28-shiomax/.jriver/Media\ Center\ 28/Temp/Delta
total 152
drwxr-xr-x 1 me 1000    248 Jul  7 11:24 ./
drwxr-xr-x 1 me 1000  14576 Jul  7 11:29 ../
-rw-r--r-- 1 me 1000    574 Jul  7 11:24 browser.jmd
-rw-r--r-- 1 me 1000   1750 Jul  7 11:24 field\ (date\ tagged).jmd
-rw-r--r-- 1 me 1000 113178 Jul  7 11:24 field\ (library\ merge\ info).jmd
-rw-r--r-- 1 me 1000    547 Jul  7 11:24 mediafiles.jmd
-rw-r--r-- 1 me 1000     16 Jul  7 11:24 platform.jmd
-rw-r--r-- 1 me 1000  14926 Jul  7 11:24 playlistx.jmd
-rw-r--r-- 1 me 1000     16 Jul  7 11:24 removable.jmd
-rw-r--r-- 1 me 1000    283 Jul  7 11:24 user.jmd

After changing the variable to 100 I get

Code: [Select]
root@me-server:~# ls -la /mnt/user/appdata/jriver28-shiomax/.jriver/Media\ Center\ 28/Temp/Delta
/bin/ls: cannot access '/mnt/user/appdata/jriver28-shiomax/.jriver/Media Center 28/Temp/Delta': No such file or directory

Looks like the app successfully removed it.
Logged

max096

  • MC Beta Team
  • Galactic Citizen
  • *****
  • Posts: 363
Re: Smartlist duplicated thousands of times
« Reply #7 on: July 07, 2021, 10:46:20 am »

Honestly, setting the groupid should have changed nothing at all. The group has less access on the folders than the user going off your ls output. There is no additional access you should have gotten from adding the group.

Any chance you restarted the container? Because JRiver does seem to clear the temp dir when it restarts. You donīt even need to restart the container, you can just press the X (close) in VNC and it will get restarted too. Same result.

Not sure if JRiver is supposed to periodically clear the directory. I had a lot of files in there too.

Quote
failed to commit changes to dconf

This happens when you open a file explorer inside the container. It opens nautilus and there is no gnome. Dconf is part of the gnome configuration system. Not totally sure what it wants to do with dconf. But I donīt see a reason why one would want to customize nautilus inside the container and it otherwise works fine. Just complains about it every time you open it.
Logged

merman-corrode-portage

  • Recent member
  • *
  • Posts: 31
Re: Smartlist duplicated thousands of times
« Reply #8 on: July 07, 2021, 07:40:39 pm »

Fair enough, I noticed that the Delta folder appeared again shortly after, however the error has not popped up since. I'm now getting another error related to the Temp directory   :-[

Code: [Select]
mv: cannot move '/config/.jriver/Media Center 28/Temp/JR File Loader - 22418362717952 (10000) (10000) (79).dat' to '/config/.jriver/Media Center 28/Temp/Downloaded Images/Image.jpg': No such file or directory
Logged
Pages: [1]   Go Up