Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[gappie]

ive posted about this in the past (during 15 and 16 development i think) a few times, but i can now reproduce it.  

ive had movies disappear from my database several times over the years, so i turned of autoimport, to get some insight on what happens. this morning when running autoimport i got the notification that 375 files where removed from my database. all movies on external harddrives that are off. reimported them they seems to have the last played etc again with them.

but that made me think... i run autoimport oft enough, and had not seen it for a long time, so what triggered this.

well i can reproduce now by turning on the external drive, run autoimport, turn off the external drive, run autoimport again. See attachment for what i get. the drives have : fix missing files>yes (protect missing drives).

it does not happen after shutting down mc and the library server inbetween the 2 autoimports.

 
gab

[Matt]

well i can reproduce now by turning on the external drive, run autoimport, turn off the external drive, run autoimport again. See attachment for what i get. the drives have : fix missing files>yes (protect missing drives).

This is a tricky one.

If the drive is there and the file is not, we consider the file missing.

We cache whether a drive is there for a minute or so, since the system can't always answer that question quickly (especially for network drives).

That one minute cache is causing the issue.

[fitbrit]

Oh no. That could explain the issues I posted about recently with files dropping out of my library. Unraid is having some permissions issues on one of my mc client htpcs; I can't access mounted drives until I log in whereas the mc server htpc doesn't have this issue. I will turn off sync changes for now I suppose. hmmm, on second read perhaps this isn't the cause since the drive themselves are inaccessible during this time.

[gappie]

This is a tricky one.

If the drive is there and the file is not, we consider the file missing.

We cache whether a drive is there for a minute or so, since the system can't always answer that question quickly (especially for network drives).

That one minute cache is causing the issue.

ah, ok. though i thought it was longer the first time. maybe ill test some more later. but when you run autoimport in the background, is there a change that that will happen too. i sometimes put those disks on to update some tags and then off again.
and when they get removed and 'ignore files previous removed from database' is on?
just ask because i lost the files in the lib in the past with autoimport running in the background.

thanks
 
gab

[gappie]

Oh no. That could explain the issues I posted about recently with files dropping out of my library. Unraid is having some permissions issues on one of my mc client htpcs; I can't access mounted drives until I log in whereas the mc server htpc doesn't have this issue. I will turn off sync changes for now I suppose. hmmm, on second read perhaps this isn't the cause since the drive themselves are inaccessible during this time.

just trying to think out loud.. does mc see those mounted drives, but not seeing the files before logging in? could that not be it?

 
gab

ps. turning it off could be a nice idea none the less. you can add an autoimport icon to the taskbar, and you always get a summery after running, could help to see when it happens. and you know then that they fell off.

[gappie]

tested some more. when i leave the external disk out of autoimport, and i run autoimport within 'the one minute rule', the files from that disk are deleted none the less, and ofcourse not added again when the disks are connected. (this explains why they did not get back in the past, i had autoimport just watching a few folders and used mc to place them where they belong)

an other thing is that sometimes not all files get deleted from the library, just a part. 

 
gab

[Matt]

tested some more. when i leave the external disk out of autoimport, and i run autoimport within 'the one minute rule', the files from that disk are deleted none the less, and ofcourse not added again when the disks are connected. (this explains why they did not get back in the past, i had autoimport just watching a few folders and used mc to place them where they belong)

It might be two or three minutes.  I'm going from memory.

Now where did I leave my remote?

[gappie]

It might be two or three minutes.  I'm going from memory.

Now where did I leave my remote?

ok .. so you say it is wise from me to turn of autoimport and just run it manually, and use it wisely.. i can live with that.

 
gab

[glynor]

ok .. so you say it is wise from me to turn of autoimport and just run it manually, and use it wisely.. i can live with that.

You don't have to turn off AutoImport entirely, just disable the Fix Broken Links option.

[Matt]

I'd like to solve this, but I'm not sure how to handle it properly.

I suppose the code could do something like this:

Code: [Select]

if (FileMissing)
{
    if (DriveMissing)
    {
        // since the drive is also missing, we'll assume this file is on a drive that comes and goes
    }
    else
    {
        // double-check and don't allow using cache so we never remove a file due to an out-of-state response caused by caching
        if (DriveMissing(NoAllowCache))
        {
            RemoveFile(...)
        }
    }
}

Thoughts?       

[Matt]

Next build:
Fixed: Caching of drive availability could lead to a case where auto-import would incorrectly remove a file on a hard drive that comes and goes (like a USB drive with a power switch, etc.).

I don't have an easy way to test this, so your help testing would be appreciated.

[gappie]

You don't have to turn off AutoImport entirely, just disable the Fix Broken Links option.

that is indeed an option. and make a view to check which files are missing, turn all the disk on and delete them from the base.
but then i should put them all in autoimport. because when a drive is not putted there that has files in the base, then they can get lost. you can not say to leave those files alone when running autoimport eventough those drives are not included in autoimport. they just get kicked of.

 
gab

[gappie]

Next build:
Fixed: Caching of drive availability could lead to a case where auto-import would incorrectly remove a file on a hard drive that comes and goes (like a USB drive with a power switch, etc.).

I don't have an easy way to test this, so your help testing would be appreciated.

thanks matt... i love testing 

thanks
 
gab

[glynor]

that is indeed an option. and make a view to check which files are missing, turn all the disk on and delete them from the base.

Right.  That's what I do.  If you "know" ahead of time which drives are "allowed" to have missing files you want removed (and which ones should never have missing files), it is even easier.

Interesting, Matt.  I'll try this out, though I don't know if it'll apply to my situation (almost certainly not).

For the record, I have a command line utility that is almost done that can take a Playlist or View "path" as the argument, and will go through the list and remove any missing files it finds.  It is working fine for me, but I need to finish up the command line arguments system.

[gappie]

Next build:
Fixed: Caching of drive availability could lead to a case where auto-import would incorrectly remove a file on a hard drive that comes and goes (like a USB drive with a power switch, etc.).

I don't have an easy way to test this, so your help testing would be appreciated.

nice! the build became available.. if tested the same scenarios as in the posts above and some other things i could think of. and it seems to be working fine.

thanks
 
gab