INTERACT FORUM

Windows => Plug-in Development => Topic started by: raldo on April 28, 2009, 05:06:56 pm

Title: Personal Video Database Import plugin - Test log
Post by: raldo on April 28, 2009, 05:06:56 pm
Current plugin version 1.0.13

PvdImport is currently released as beta, so expect "issues"...
-------------------------
1. About PvdImport

Personal Video Database (PVD) is a program which pulls metadata and posters for movies and TV shows from online sources (imdb, amg and others) - http://www.videodb.info/forum_en/

PvdImport helps in getting data from the PVD database into MC, allowing us to match PVD fields with fields in MC.

Here are some screenshots (courtesy of Darichman) of MC with data pulled from PVD (using AMG and IMDb info) here: http://www.pix01.com/gallery/7E547B99-5004-478D-A4AC-37AD672F51A4/MC_Gallery/bigpage.html?image=133162147_orig7.jpg

-------------------------
2. Installation

o Install the plugin msi from here (http://www.tonsbergibk.com/SetupPvdImport.msi). On Vista/Win7, you probably should run the installer as Administrator.
o You may have to install MS .net framework 2.0
o Install firebird server 2.0.5 with default settings from here (http://www.firebirdsql.org/index.php?op=files&id=engine_205)
o Install PVD 0.9.9.6 from here (http://www.videodb.info/forum_en/index.php?topic=1294.0)

------------------
3.1 PVD setup

o from the file menu, select connect to server
o enter the path to your PVD database file (usually under C:\Documents and Settings\[user]\My Documents\Personal Video Database)

Notes on DVD handling: DVD files (.ifo, .vob, etc.) are have [Filename] that ends with .dvd;1 internally in MC. The corresponding filename in PVD must end with .ifo for PvdImport to work. 

------------------
3.2 PvdImport setup

o In Media Center, goto "Services & plugins" and select PvdImport
o The lowermost area in the plugin window shows status texts
o You may need to set the folder of the PVD database
o Set your preferred cover art folder mode
o Set your cover art folder

4. Using PVD Import
-------------------

There are basically three ways you can update fields: Test, from the MC tree using "Send To", and from the Plugin (mass import).

Typically, you would test the plugin the first time you use it to see if your setup is correct. Then, you would do a mass import.

Later, you would do imports from the tree since there probably will be fewer files to import.

It might be a good idea to backup your database before you start testing.

------------------
4.1 PvdImport Test

o Press "set file" and select one of your video files that you know exists in the PVD database.
o Press "test" (Warning: pressing "Test will update your library too)
o The fields in the "SqlResult" should now be populated by data.

------------------
4.2 PvdImport from the MC tree

o Select one or several files (Currently, about 50 files is the maximum you can update in one batch)
o Right click and navigate to "Send to external" in the context menu
o PvdImport should already be an entry in this menu.
o After you have selected "PvdImport", the fields of your selected files should be populated with data.

------------------
4.3 Mass Import

o PvdImport uses an MC Search Expression to select the files that will be updated when you do a mass import.
o You can use search expressions to pick playlists, all video files, etc.
o You can make your own search expression using the wizard in the MC search field (upper right corner of mc).
o Afterwards, change to "Text" in the search context menu and you will see the expression
o Example: All video, tv, data : [Media Type]=[Data],[TV],[Video]
o Enter this search expression in the Edit box to the right of the Import button
o Press the import button and a progress bar + text will indicate progress.
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on April 28, 2009, 05:51:06 pm
*********************
Known Issues 1.0.13:
*********************
o If you select too many files (in the hundreds) in "Send To...", the PvdImportContextMenu.exe applications may get confused. Select fewer (50, maybe)!

1.0.13:
----------
o Changed: Updated field configuration to contain Darichman's suggestions too.
o Feature: New checkbox "Ignore". When selected all fields marked Ignore are displayed, otherwise, none marked ignore are displayed.

1.0.12:
----------
o Changed: Some more comments in the user manual
o Changed: Several new fields/rows and corresponding sql+description, thanks to rick.ca
o Feature: New Description column in Field Configuration: Contains description of rows
o Feature: New checkbox which allows the user to hide Description column
o Changed: Some cosmetics in field configuration + corresponding tooltip changes
o Changed: "Sucess" -> "Success" :)
o Changed: Internal stuff: Databinding of all controls now works according to how it's supposed to
o Changed: Internal stuff: Split view code from data handling code. Better structure of data handling code

1.0.11:
----------
o Fixed: Runtime information in lower Listbox. Added some logging information.
o Fixed: There is a memory leak in the Firebird server. This memory leak is related to PvdImport sql reads.

1.0.10:
----------
o Fixed: Test filename would occasionally be truncated so that no data was imported

1.0.9:
----------
o feature: Mass import Stop Button
o feature: Removed Indexing of lists.
o Feature: Software User Manual (SUM) is now included in the installation. A button (SUM) invokes Notepad.exe which loads the file.
o Fixed: Some drawing issues when resizing MC.
o Fixed: Tooltip for field configuration headers doesn't show properly. You can deselect tooltips with a checkbox
o Fixed: Don't push test button until sql results are ready, the application will hang
o Fixed: User information (lower window) is not updated properly during import.

1.0.8:
----------
o feature: Mass import, uses MC expressions. See User manual for details.
o feature: Write only empty fields
o Improved: Thread handling when importing many files.
o Feature: Progress bar and current import status text
o Improved: Some changes in database handling (open/close)
o Improved: Internal code structure

1.0.7
-----
o Fixed: Plugin doesn't handle DVDs (.ifo, .vob, etc.)
o Fixed: folder.jpg for cover art doesn't work properly.
o Fixed: If a movie filename and or path contains a special character like ' or " (umlaut) then the data will not be imported from pvd.

1.0.6
-----
o Fixed by removing plugin skinning: Critical parts of the plugin are not visible in the ThunderStorm or Blue Steel skins.
o Fixed: An empty MC field leads to an exception

1.0.5
-----
o Cover art Functionality. "Emulates" the setup of the tools/options dialog in MC -- There is no API accessible in plugins for this).
o New Ignore field: Selected fields items will not be updated

1.0.4
-------
o Change installer so that you don't have to uninstall previous version. Thus the "new" versioning
o Change installer so that .xml files from previous version are preserved
o Database location was hardcoded
o save button was removed by error in previous version

0.0.0.3:
-------
o Set default config/pvd file locations when the stored filepaths aren't found
o Avoid additional test button pushes when test is in progress.

0.0.0.2:
-------
o Default install location didn't work properly
o Text boxes for file paths are readonly
o Removed "save" buttons, this is done when file is ok'ed
o Saved file locations weren't updated properly to text boxes.

Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on April 29, 2009, 12:25:01 am
Quote
1. Installation
o Install the plugin msi from here

The installation appears to use a fixed path. It didn't detect my plugins directory, which is C:\Program Files\Media Center 13\Plugins (i.e., no "J River").

Quote
2.1 PVD setup
o from the file menu, select connect to server

I've reported (http://www.videodb.info/forum_en/index.php?topic=1260.msg4890#msg4890) a PVD bug concerning this. I'm sure it will be fixed soon. As you know, the link you have provided should still be good after the fix is posted.

Quote
2.2 PvdImport setup
o In Media Center, goto "Services & plugins" and select PvdImport

Critical parts of the plugin are not visible in the ThunderStorm (my favorite) or Blue Steel skins.

Quote
o The lowermost area in the plugin window shows status texts

Regardless of my configuration settings (more on that below), mine shows:

Plugin Initiated OK
Failed to open database (when retrieving filenames);I/O error for file CreateFile (open) "C:\Documents and Settings\Harald\My Documents\Personal Video Database\movies.pvd"
Error while trying to open file

It seems your own path is hard-coded in PvdImport.dll, and the configuration settings are ignored.

The set database and set test file buttons don't work. I had to cut & paste file paths into the boxes. Attempting to save the configuration causes an exception...

Code: [Select]
************** Exception Text **************
System.NullReferenceException: Object reference not set to an instance of an object.
   at PvdImport.MainInterface.buttonSaveConfiguration_Click(Object sender, EventArgs e)
   at System.Windows.Forms.Control.OnClick(EventArgs e)
   at System.Windows.Forms.Button.OnClick(EventArgs e)
   at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
   at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
   at System.Windows.Forms.Control.WndProc(Message& m)
   at System.Windows.Forms.ButtonBase.WndProc(Message& m)
   at System.Windows.Forms.Button.WndProc(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)


************** Loaded Assemblies **************
mscorlib
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.1433 (REDBITS.050727-1400)
    CodeBase: file:///C:/WINDOWS/Microsoft.NET/Framework/v2.0.50727/mscorlib.dll
----------------------------------------
PvdImport
    Assembly Version: 1.0.3405.42397
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/Media%20Center%2013/Plugins/PvdImport/PvdImport.dll
----------------------------------------
System.Windows.Forms
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.1433 (REDBITS.050727-1400)
    CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Windows.Forms/2.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.1433 (REDBITS.050727-1400)
    CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System/2.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
System.Drawing
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.1433 (REDBITS.050727-1400)
    CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Drawing/2.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
DataConfig
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/Media%20Center%2013/Plugins/PvdImport/DataConfig.DLL
----------------------------------------
Accessibility
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.1433 (REDBITS.050727-1400)
    CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/Accessibility/2.0.0.0__b03f5f7f11d50a3a/Accessibility.dll
----------------------------------------
System.Xml
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.1433 (REDBITS.050727-1400)
    CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Xml/2.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------
NamedPipe
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/Media%20Center%2013/Plugins/PvdImport/NamedPipe.DLL
----------------------------------------
System.Configuration
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.1433 (REDBITS.050727-1400)
    CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Configuration/2.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll
----------------------------------------
gyc8vepr
    Assembly Version: 1.0.0.0
    Win32 Version: 2.0.50727.1433 (REDBITS.050727-1400)
    CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System/2.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
3whkwwr-
    Assembly Version: 1.0.0.0
    Win32 Version: 2.0.50727.1433 (REDBITS.050727-1400)
    CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System/2.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
Interop.MediaCenter
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/Media%20Center%2013/Plugins/PvdImport/Interop.MediaCenter.DLL
----------------------------------------
FirebirdSql.Data.FirebirdClient
    Assembly Version: 2.0.1.0
    Win32 Version: 2.0.1.0
    CodeBase: file:///C:/Program%20Files/Media%20Center%2013/Plugins/PvdImport/FirebirdSql.Data.FirebirdClient.DLL
----------------------------------------
System.Data
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.1433 (REDBITS.050727-1400)
    CodeBase: file:///C:/WINDOWS/assembly/GAC_32/System.Data/2.0.0.0__b77a5c561934e089/System.Data.dll
----------------------------------------
System.Transactions
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.1433 (REDBITS.050727-1400)
    CodeBase: file:///C:/WINDOWS/assembly/GAC_32/System.Transactions/2.0.0.0__b77a5c561934e089/System.Transactions.dll
----------------------------------------

Thinking this was preventing a connection, I edited PvdImportConfiguration.xml manually, and restarted MC, but this made no difference—the status area still indicates your path is being used, and I'm unable to connect.

BTW, using the same buttons to save both configuration files makes it too easy to overwrite the wrong file.
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on April 29, 2009, 01:20:30 am
It's alpha—I shouldn't give up so easily...  :-\

I've moved my database to C:\Documents and Settings\Harald\My Documents\Personal Video Database\MOVIES.PVD and revised PvdImportConfiguration.xml accordingly. I don't get the "Failed to open database" message, but "Test" crashes MC. The log:

Code: [Select]
Media Center; Version: 13.0.152; Types: 16383
0020009: 5236: General: CMCResourceHelper::GetIsModalPopupShowing: Menu showing
0022472: 5236: General: CMCResourceHelper::GetIsModalPopupShowing: Menu showing
0034369: 5236: General: CMCResourceHelper::GetIsModalPopupShowing: Main window disabled
0034439: 5236: General: RunProgram: Start
0034479: 5236: General: RunProgram: Filename: C:\Documents and Settings\Rick\Application Data\J River\Media Center 13\Log.txt / Parameters:
0034579: 5236: General: RunProgram: Performing ShellExecute...
0034750: 5236: General: RunProgram: Running process...
0034810: 5236: General: RunProgram: Finish (371 ms)
0046296: 5236: Podcast: CPodcastScheduler::Update: Start
0046336: 5236: Podcast: CPodcastScheduler::Update: Processing completed work threads
0046527: 5236: Podcast: CPodcastScheduler::Update: Purging files
0046587: 5236: Podcast: CPodcastScheduler::Update: Scheduling downloads
0046687: 5236: Podcast: CPodcastScheduler::ScheduleDownload: Start
0046727: 5236: Podcast: CPodcastScheduler::ScheduleDownload: No work found
0046767: 5236: Podcast: CPodcastScheduler::ScheduleDownload: Finish (80 ms)
0046787: 5236: Podcast: CPodcastScheduler::Update: Finish (491 ms)
0052345: 5236: General: CMCResourceHelper::GetIsModalPopupShowing: Main window disabled
0053677: 5236: General: CMCResourceHelper::GetIsModalPopupShowing: Main window disabled
0069690: 5668: General: TopLevelExceptionFilter: Unhandled exception -- program crashing
0069770: 5668: General: TopLevelExceptionFilter: Message: 0, wParam: 0, lParam: 0, Window class:

Okay, now I give up. ;)
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on April 29, 2009, 01:53:04 am
Thanks for asking :)

I'm giving it a go now...
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on April 29, 2009, 02:05:20 am
Okay, now I give up. ;)

That's understandable :) By the way, can you see PvdImport in "Send to external"? I'm just wondering (if at least) the registry setting is correct...
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on April 29, 2009, 03:31:09 am
My observations...

Set up Test PVD Database
D:\Tools\Personal Video Database\Ramus Test Install\Test Library\Ramus Test Library.pvd

Dummy Movie Files Created and Imported into PVD. Fields populated with IMDb data.
Same files imported into MC library.

Plugin msi installed in MC default plugin directory

Accessed plugin from Services & Plugins within MC

Bug - Despite Setting the PVD database as "Ramus Test Library" I kept getting the following error:

Code: [Select]
Warning!! Failed to find Pvd database (C:\Documents and Settings\Harald\My Documents\Personal Video Database\movies.pvd)
Plugin Initiated OK

I manually entered Paths for PVD DB and Video File.
Unlike Rick, I got no error on saving the config file.

Upon exiting the plugin and re-entering, the config file had reset to the default.
"Loading" my previously saved config file "Chris.xml" failed to update the field.
I manually entered "C:\Program Files\J River\Media Center 13\Plugins\PvdImport\Chris.xml"

Pressing Test has no effect, but I got the following error:

Code: [Select]
Failed to open database (when retrieving filenames);I/O error for file CreateFile (open) "C:\Documents and Settings\Harald\My Documents\Personal Video Database\movies.pvd"
Error while trying to open file
Failed to open database (when retrieving filenames);I/O error for file CreateFile (open) "C:\Documents and Settings\Harald\My Documents\Personal Video Database\movies.pvd"
Error while trying to open file

So yes, it seems despite updating all the relevant fields manually, the plugin keeps defaulting to your PVD database location :)

Looks very promising though! Anything we can do, let us know.

I'll keep fiddling.
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on April 29, 2009, 04:02:58 am
Success ;D

I created a test library at C:\Documents and Settings\Harald\My Documents\Personal Video Database\movies.pvd and everything worked as expected from there. I wonder what happened with yours, Rick? Weird.
The buttons still don't work quite right (eg Set fails to update the video file) but Test worked for me:

(http://www.pix01.com/gallery/7E547B99-5004-478D-A4AC-37AD672F51A4/PVD_Import_Plugin/1350142020.jpg)
Click to enlarge. (http://www.pix01.com/gallery/7E547B99-5004-478D-A4AC-37AD672F51A4/PVD_Import_Plugin/bigpage.html?image=135014202_orig0.jpg)

I also tried the "Send to External -> PVDImport" which worked just as you described for the two dummy files I had:

(http://www.pix01.com/gallery/7E547B99-5004-478D-A4AC-37AD672F51A4/PVD_Import_Plugin/1350142021.jpg)
Click to enlarge. (http://www.pix01.com/gallery/7E547B99-5004-478D-A4AC-37AD672F51A4/PVD_Import_Plugin/135014202_orig1.jpg)

I think we have a winner!
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on April 29, 2009, 04:16:22 am


I think we have a winner!

Indeed...this sounds as the way to go for me too...I like pvd much better then mymovies.

Mabe the plugin could be extended in the future to have an option to write mc fields info to the pvd database then we even would have a neat two way solution.

peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on April 29, 2009, 04:25:16 am
I'll keep fiddling.
Great, I'll try and fix those issues with the paths later on today.

Rick.ca, do you see the "send to external" item?

Which OS are you guys on?
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on April 29, 2009, 05:24:15 am
I'm on Windows XP Media Center Edition, SP3
MC 13.0.160
PVD 0.9.9.5 (Nostra's latest beta, AFAIK)
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on April 29, 2009, 05:59:35 am
Quote
I wonder what happened with yours, Rick? Weird.

It works! 8)

I tried your approach of trying a small test database. It works fine, other than the fixed directory and other minor glitches mentioned.

I'll try my usual database again tomorrow. Maybe the combination of the size of my database (1,600+ movies, 567 Mb), my tired old computer and my lack of patience is the problem. ::)

Quote
Rick.ca, do you see the "send to external" item?

Yes, although I had to change the path (again, to remove the "J River").

Quote
Which OS are you guys on?

XP.

Good work! Now finish it up quick. Now I'm eager for phase 2—getting all this information displayed in Theater View. :)
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on April 29, 2009, 01:30:20 pm
Quote
...my lack of patience is the problem.

That was it. It takes more than 10 seconds to query a record in my full database. Hitting "Test" while that's in process causes MC to freeze. I don't know if it indicates why, but following is a log of me starting a query and then clicking "Test" 6 seconds later—before the query was complete. MC then becomes unresponsive (the mouse cursor becomes an hour glass), although it can be closed from the taskbar. On restart, MC reports an improper shut-down.

Code: [Select]
Media Center; Version: 13.0.152; Types: 16383
0013780: 924: General: CMCResourceHelper::GetIsModalPopupShowing: Menu showing
0013900: 924: General: RunProgram: Start
0013960: 924: General: RunProgram: Filename: C:\Program Files\Media Center 13\Plugins\PvdImport\PvdImportContextMenu.exe / Parameters: "L:\Oscars\Babette's Feast (1987) 7.7.avi"
0014000: 924: General: RunProgram: Performing ShellExecute...
0014140: 924: General: RunProgram: Running process...
0014271: 924: General: RunProgram: Finish (371 ms)
0014391: 924: Reader: CLocalReader::OpenInternal: Opening: C:\Documents and Settings\Rick\Application Data\J River\Media Center 13\Library\links.xml
0014551: 924: Reader: CLocalReader::Close: Closing: C:\Documents and Settings\Rick\Application Data\J River\Media Center 13\Library\links.xml
0014601: 924: Database: CMJSearchHelper::GetResults: Search: [media type]=audio; Elapsed ms: 0.428
0014771: 924: Database: CMJSearchHelper::GetResults: Search: [Media Type]=[Audio],[Video]; Elapsed ms: 0.489
0014861: 924: Database: CMJSearchHelper::GetResults: Search: [Media Type]=[Audio]; Elapsed ms: 0.427
0014922: 924: Database: CMJSearchHelper::GetResults: Search: [Media Type]=[Video]; Elapsed ms: 0.429
0014972: 924: Database: CMJSearchHelper::GetResults: Search: ; Elapsed ms: 0.198
0020760: 924: Reader: CLocalReader::OpenInternal: Opening: C:\Documents and Settings\Rick\Application Data\J River\Media Center 13\Library\links.xml
0020810: 924: Reader: CLocalReader::Close: Closing: C:\Documents and Settings\Rick\Application Data\J River\Media Center 13\Library\links.xml
0020830: 924: Database: CMJSearchHelper::GetResults: Search: [media type]=audio; Elapsed ms: 0.463
0020870: 924: Database: CMJSearchHelper::GetResults: Search: [Media Type]=[Audio],[Video]; Elapsed ms: 0.520
0020890: 924: Database: CMJSearchHelper::GetResults: Search: [Media Type]=[Audio]; Elapsed ms: 0.462
0020910: 924: Database: CMJSearchHelper::GetResults: Search: [Media Type]=[Video]; Elapsed ms: 0.463
0020930: 924: Database: CMJSearchHelper::GetResults: Search: ; Elapsed ms: 0.249
0021000: 924: General: CMainUIWnd::Resize: Start
0021080: 924: General: CMainUIWnd::Resize: Finish (80 ms)
0000151: 4196: Reader: CLocalReader::OpenInternal: Opening: C:\Program Files\Media Center 13\Data\Languages\English (Canadian)\Language.txt
0000221: 4196: Reader: CLocalReader::Close: Closing: C:\Program Files\Media Center 13\Data\Languages\English (Canadian)\Language.txt
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on April 29, 2009, 04:03:34 pm
That was it. It takes more than 10 seconds to query a record in my full database. Hitting "Test" while that's in process causes MC to
Strange, PVD doesn't respond that slowly? What about the other client you installed? I've got maybe 500 files in my collection and one query takes approximately .3 seconds.

I've fixed up some minor issues with the file paths in the new version.
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on April 29, 2009, 08:41:43 pm
Nice work Raldo :) I'm off to work, but will do some more playing around tonight.
Cheers
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on April 29, 2009, 08:45:52 pm
Quote
Strange, PVD doesn't respond that slowly? What about the other client you installed? I've got maybe 500 files in my collection and one query takes approximately .3 seconds.

Whether or not other things are connected makes no difference. I think it's just the slow machine. PVD is not as slow, but nothing happens there in .3 seconds either. :(

Quote
I've fixed up some minor issues with the file paths in the new version

What did you change? I still have to use the fixed path, which is wearing thin.

There is one significant change in the wrong direction—the field configuration can no longer be saved. The button remains as "Load fields," no matter what changes have been made. Now I really hate the "one-button-does-all" idea. ;)

Is it really necessary the previous version be uninstalled before installing a new version. It would be so much faster just to overwrite. In any case, something should be done to preserve the user's existing configuration files, or users should be advised to not use the default files for custom configurations (which, surely, most will be).
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on April 30, 2009, 01:13:22 am
Strange, PVD doesn't respond that slowly? What about the other client you installed? I've got maybe 500 files in my collection and one query takes approximately .3 seconds.

I've fixed up some minor issues with the file paths in the new version.

I think there are still a few path issues. I pointed the plugin to my .pvd. But doing Test gave this error:

Failed to open database (when retrieving filenames);I/O error for file CreateFile (open) "C:\Documents and Settings\Harald\My Documents\Personal Video Database\movies.pvd"

Ie it still looks for your hardcoded path somewhere.

peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on April 30, 2009, 02:30:57 am

There is one significant change in the wrong direction—the field configuration can no longer be saved. The button remains as "Load fields," no matter what changes have been made. Now I really hate the "one-button-does-all" idea. ;)
Yes, that was just a lapse of reason.
 
Is it really necessary the previous version be uninstalled before installing a new version. It would be so much faster just to overwrite. In any case, something should be done to preserve the user's existing configuration files, or users should be advised to not use the default files for custom configurations (which, surely, most will be).
I agree with you here.
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on April 30, 2009, 02:33:30 am
Failed to open database (when retrieving filenames);I/O error for file CreateFile (open) "C:\Documents and Settings\Harald\My Documents\Personal Video Database\movies.pvd"
Hey Peter, thanks for joining in!

Yes, this particular issue disappeared in the noise. I'll fix it in the next update.
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on April 30, 2009, 05:56:55 am
No luck for me with using a custom database file:

Code: [Select]
Failed to open database (when retrieving filenames);I/O error for file CreateFile (open) "D:\Tools\Personal Video Database\Ramus Test Install\Test Library\Ramus Test Library.PVD"
Error while trying to open file

The Load/Save & Set Database buttons all seem to work as expected now though. Exiting MC and loading a saved config file loads all the relevant paths again which is nice.
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on April 30, 2009, 06:06:38 am
No luck for me with using a custom database file:

Code: [Select]
Failed to open database (when retrieving filenames);I/O error for file CreateFile (open) "D:\Tools\Personal Video Database\Ramus Test Install\Test Library\Ramus Test Library.PVD"
Error while trying to open file

Can you ckeck if this file is locked by PVD maybe? Try closing down PVD and hit "test" again; you might still be using PVD through the embedded client (which locks down the database).

I'm at work now and I don't have my code at hand, but I'm pretty sure that PvdImport fails in *opening* the file (I.e. PvdImport finds the file).

There is a problem in beta4 of PVD which I've experienced, so for now, I cannot run PVD simultaneously with PvdImport (or Firebird Maestro).


The Load/Save & Set Database buttons all seem to work as expected now though. Exiting MC and loading a saved config file loads all the relevant paths again which is nice.
Good.
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on April 30, 2009, 06:29:59 am
Can you ckeck if this file is locked by PVD maybe? Try closing down PVD and hit "test" again; you might still be using PVD through the embedded client (which locks down the database).

I'm at work now and I don't have my code at hand, but I'm pretty sure that PvdImport fails in *opening* the file (I.e. PvdImport finds the file).

There is a problem in beta4 of PVD which I've experienced, so for now, I cannot run PVD simultaneously with PvdImport (or Firebird Maestro).

Good.

Was just posting :) Yes, I tried again after closing PVD and the sql results were populated. I noticed that "Test" actually updates the fields in MC as well - this was not expected.
Update of fields via Send To -> PVDImport is also working fine, just as before.

I know we're probably not at graphical interface level of design yet, but the mouseover popups of fields/column headers from within the plugin view (in Media Center) are unreadable. I'm using the default (Noire) skin.

I can start testing out custom fields and a larger library now.

Two questions though... how does the numbered list column work? The checkbox turns it on? Is this an optional solution for preserving artists etc as numbered lists? I'll have to consider this. As you you guys might be aware, MC only alphabetises the field if it is manually edited, so provided you don't touch the entries once they're in MC, it may not be necessary. It's an interesting idea though...

I'll have to examine the language in the PVDexpression part of the plugin a bit more closely. What would the general template for a custom field be?
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on April 30, 2009, 06:42:20 am
Was just posting :) Yes, I tried again after closing PVD and the sql results were populated. I noticed that "Test" actually updates the fields in MC as well - this was not expected.
Update of fields via Send To -> PVDImport is also working fine, just as before.
Good point, we might turn off update when "test" is pushed. Did you make a backup of your library, btw. :)

I know we're probably not at graphical interface level of design yet, but the mouseover popups of fields/column headers from within the plugin view (in Media Center) are unreadable. I'm using the default (Noire) skin.
I know, and I've tried googling this but to no avail..

I can start testing out custom fields and a larger library now.

Two questions though... how does the numbered list column work? The checkbox turns it on? Is this an optional solution for preserving artists etc as numbered lists. I'll have to consider this. As you you guys might be aware, MC only alphabetises the field if it is manually edited, so provided you don't touch the entries once they're in MC, it may not be necessary. It's an interesting idea though...
Yes. Check-> On. Rick.ca desperately wants this :)  For me it's not an issue. You can just deselect the option.

I'll have to examine the language in the PVDexpression part of the plugin a bit more closely. What would the general template for a custom field be?

It's Sql, so fairly cryptic. But very flexible. If you compare,say,  "Author" and "Screenwrite" you'll see a "general template". There is a system, or template if you like, but it's dictated by the PVD database structure.

I suggest you download Firebird Maestro and experiment if you would like some other fields populated, etc.
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on April 30, 2009, 07:20:29 am
Hey Peter, thanks for joining in!

Yes, this particular issue disappeared in the noise. I'll fix it in the next update.

Ok 1.0.4 works as expected.

I did a test with 25 movies. From these 12 had entries in the pvd database. It worked correctly with the right click send to method, those with entries in pvd got updated the others had no changes. Maybe an option could be added to have a popup message with some useful info on the update. Should be an option else it would be annoying...but with a large number of files it would be useful to have...and maybe even an a message like processing x out of total y records...if a large batch takes longer then a few seconds...else people might think the computer hangs.

Thanks for a really useful plugin!  8)


peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: Mr ChriZ on April 30, 2009, 07:40:52 am
Sorry could someone explain what this plugin is for?
 :)
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on April 30, 2009, 08:22:59 am
Sorry could someone explain what this plugin is for?
 :)

Personal Video Database (PVD) is a program which pulls metadata/posters for movies and TV shows from online sources (imdb, amg and others) - http://www.videodb.info/forum_en/ (http://www.videodb.info/forum_en/)

This plugin that raldo is developing should help in getting data from the PVD database into MC, allowing us to match PVD fields with fields in MC.

Edit: I have some screenshots of MC with data pulled from PVD (using AMG and IMDb info) here: http://www.pix01.com/gallery/7E547B99-5004-478D-A4AC-37AD672F51A4/MC_Gallery/bigpage.html?image=133162147_orig7.jpg (http://www.pix01.com/gallery/7E547B99-5004-478D-A4AC-37AD672F51A4/MC_Gallery/bigpage.html?image=133162147_orig7.jpg)
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on April 30, 2009, 08:39:30 am
I found one issue. If a movie filename and or path contains a special character like ' or " (umlaut) then the data will not be imported from pvd.

peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on April 30, 2009, 09:01:05 am
I found one issue. If a movie filename and or path contains a special character like ' or " (umlaut) then the data will not be imported from pvd.

peter
Yes, I saw that. It's noted.

Peter, can you test the following, please:
o Uninstall PvdImport.
o Delete the remaining folder in the plugin directory.
o Reinstall.

I'd like to see if a clean install manages to find all te files. Especially the default PVD database location found under my documents.
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on April 30, 2009, 09:19:37 am

I'd like to see if a clean install manages to find all te files. Especially the default PVD database location found under my documents.

Ok :)

Yes it did find:
C:\Users\Peter\Documents\Personal Video Database\movies.pvd

That one exists, but I actually use:
C:\Users\Peter\Documents\VIDEO DB.PVD

Using set I select it and it then works ok.

peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on April 30, 2009, 09:25:35 am
Ok :)

Yes it did find:
C:\Users\Peter\Documents\Personal Video Database\movies.pvd

That one exists, but I actually use:
C:\Users\Peter\Documents\VIDEO DB.PVD

Using set I select it and it then works ok.

peter
Great!
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on April 30, 2009, 09:39:08 am
Great!

Indeed.

I changed in the plugin settings mc [Original Title] to mc [Name] for the title import and that worked as expected :) Maybe I will try to change it to add the pvd translated title when it exist as well...but my sql is very rusty ;) Anyway after you got the special characters thingy fixed I am all set. In the meantime I'll have some "fun" entering all my movies into pvd ;)

peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on April 30, 2009, 09:47:44 am
Indeed.

I changed in the plugin settings mc [Original Title] to mc [Name] for the title import and that worked as expected :) Maybe I will try to change it to add the pvd translated title when it exist as well...but my sql is very rusty ;) Anyway after you got the special characters thingy fixed I am all set. In the meantime I'll have some "fun" entering all my movies into pvd ;)

peter

Well, PvdImport does an internal search for filename and sees if the filename is present in the PVD database. If it is present, data is imported.

Have fun!
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on May 01, 2009, 03:02:48 am
Any ideas how to get this working with ripped dvd's? With mc you cant just point pvd to the .ifo file in the VIDEO_TS folder because mc creates its unique filename in the filename field (ie .dvd;1) Actually I would prefer if I could just use .ifo with mc, but I have not seen an option in mc to do just that.

peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 01, 2009, 06:55:40 am
There is a problem in beta4 of PVD which I've experienced, so for now, I cannot run PVD simultaneously with PvdImport (or Firebird Maestro).

If this is the bug I reported, it shouldn't be stopping you from running PVD and PvdImport simultaneously. If PVD is "stuck" running Firebird Embedded, edit your pvdconfig.ini so the server connection will be used. Mine looks like this:

Code: [Select]
[DataBase]
FileName=rix:C:\Data\PVD\MY MOVIES BETA 5.PVD
Mode=0
RecentFiles=..\..\Data\PVD\Dummy.PVD|rix:C:\Data\PVD\TEST.PVD
Network=1
RecentTypes=0;1;

I've been spending a lot of time with this over the last 36 hours. That's not because of problems with the plugin—it (0.0.0.3) has been working well. I've been doing a lot of fiddling to set up dummy media files so I can get information about non-owned media into MC as well (most of my PVD database consists of movies I've seen, but do not necessarily own).

Actually, my problems started with recently with 1.0.4. I was able to get my database moved back to it's normal location, and this reflected in the configuration, but experienced exceptions (like before, I think), crashes (no cause reported by MC), and the plugin did not seem to complete the test successfully. Finally, not wanting to do something else to create a crash, I just ignored the configuration and tried an import. It worked fine. :-\

Quote
Don't push test button until sql results are ready, the application will hang

I should have read that first. I'm not sure, but that might have been part of the problem.

Quote
If a movie filename and or path contains a special character like ' or " (umlaut) then the data will not be imported from pvd.

Yes, I discovered this. Just to be clear, "special" includes normal accented characters. BTW, PVD's title matching scanner can do replacements, so problem characters like ": " in titles can be replaced with " - " in filenames and the match still made.

The biggest problem I had I haven't yet tried with 1.0.4, but I don't image anything has changed... Trying to update too many movies at once overwhelmed my machine. It seems MC dutifully executes all the Sendto commands at once, and each one is a separate process. I find I can do 100 at a time, but over 150 is likely to cause out of memory errors, and then (especially with a few hundred processes running) everything goes to hell.

Is there a way you can limit the number of processes started at one time? I wonder if even on a much more capable machine this wouldn't result in more efficient processing. In other words, it too many resources are used managing a large number of processes, is there enough left for the server, or does the whole thing bog down?

Quote
Yes. Check-> On. Rick.ca desperately wants this.  For me it's not an issue. You can just deselect the option.

So now you tell me, when I'm half done! My anxiety over this diminished somewhat when you pointed out the order won't change unless the field is edited. I have yet to imagine why I might be editing something that's already there. But if it does become necessary... do the entries have to be numbered, or is their order actually controlled by an index? I only suggested the numbering if the only way to keep them in order was to include that in the data.

BTW, I'm putting [actor] - [role] in Actors, using...

Code: [Select]
Select PEOPLE."name" || ' - ' || CAREER."role"
From PEOPLE Inner join CAREER on (CAREER."pid" = PEOPLE."pid")
Where CAREER."ctype" = 0 And CAREER."mid" = @id

What would the general template for a custom field be?

Here's an example...

Code: [Select]
Select CUSTOM_VALUES_LONGSTR."value"
From MOVIES
Inner join CUSTOM_VALUES_LONGSTR on MOVIES."mid" = CUSTOM_VALUES_LONGSTR."mid"
Inner join CUSTOM_FIELDS_MOVIES on CUSTOM_FIELDS_MOVIES."cmfid" =  CUSTOM_VALUES_LONGSTR."cmfid"
Where MOVIES."mid"=@id and CUSTOM_FIELDS_MOVIES."dispname" = 'Review'

Note different tables are involved, depending on the data type (this one being a memo field for Review).

Quote
I suggest you download Firebird Maestro and experiment if you would like some other fields populated, etc.

This is what I'm doing, using Flyspeed SLQ Query. I still don't have a clue what I'm doing, but seem to be getting results. But I believe I've explained my obsessive trial and error methods to you before. ;D

While I'm on the subject of queries, here's Season...

Code: [Select]
Select EPISODES."season"
From MOVIES Inner join EPISODES on MOVIES."epid" = EPISODES."epid"
Where MOVIES."mid"=@id

and Episode...

Code: [Select]
Select EPISODES."epnum"
From MOVIES Inner join EPISODES on MOVIES."epid" = EPISODES."epid"
Where MOVIES."mid"=@id

...which I combine in an Expression Column with If(IsEmpty([Episode]),,SPadNumber([Season],2) EPadNumber([Episode],2)). I generally watch and delete episodes, so I figure that's good enoungh. But...

I haven't been able to figure out is how to get the series title to go with each episode. :(

Any ideas how to get this working with ripped dvd's? With mc you cant just point pvd to the .ifo file in the VIDEO_TS folder because mc creates its unique filename in the filename field (ie .dvd;1)

I don't do DVD, so I can only ask a dumb question... What would MC do if you changed the .ifo to .dvd;1 and associated that extension with MC? I expect it would not recognize it as a valid file type. But if by some miracle it does otherwise, then PVD won't object to .dvd;1 being set as a video file type. Otherwise, all I can say is "uh-oh." :-\
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 01, 2009, 07:19:15 am
That was an epic post, Rick ;) I've been doing some reading and playing with Firebird just to get a better understanding of what's going on in the background. Unfortunately I'll be away for a few days and without internet (it's a long weekend over here) so will get back to things on tues :)

I like where this is going. It's very flexible and, once set up, much (much) more convenient than creating/importing export templates. Thanks again raldo.
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 01, 2009, 08:03:46 am
Quote
I like where this is going.

So do I. It looks very promising.

Have a nice (without internet?! :o) weekend. It's okay if you think about SLQ queries while you're gone—as long as you don't say anything out loud. ;)
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on May 01, 2009, 12:20:36 pm
Ok I also added a few more fields to import, including imdb url > mc web url and length > mc custom field length.
This also worked ok...nice  :)

peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 02, 2009, 12:32:42 am
Trying to update too many movies at once overwhelmed my machine. It seems MC dutifully executes all the Sendto commands at once, and each one is a separate process. I find I can do 100 at a time, but over 150 is likely to cause out of memory errors, and then (especially with a few hundred processes running) everything goes to hell.

FWIW, here's a handful of error messages I was able to capture from the large number that are generated when this happens. It seems 100 at a time is too much too, so now I'm down to 50. The queries are also taking about 10 seconds each, so it's really not working very well for me.  :'(

Code: [Select]
PvdImportContextMenu.exe - Application Error
Application has generated an exception that could not be handled.
Process ID=0x1cd8 (7384), Thread ID=0x1b50 (6992).
---------------------------
C:\Program Files\Media Center 13\Plugins\PvdImport\\PvdImportContextMenu.exe
Attempt to access invalid address.
---------------------------
PvdImportContextMenu.exe - Application Error
---------------------------
Application has generated an exception that could not be handled.
Process ID=0xc20 (3104), Thread ID=0xcbc (3260).
---------------------------
PvdImportContextMenu.exe - .NET Framework Initialization Error
C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll could not be loaded

Quote
I haven't been able to figure out is how to get the series title to go with each episode.

At the expense of brain cells I'm sure others could tolerate losing better than me (the vacationing darichman comes to mind), I was finally able to figure this out...

Code: [Select]
Select MOVIES1."title"
From MOVIES
Inner Join EPISODES On MOVIES."epid" = EPISODES."epid"
Inner Join MOVIES MOVIES1 On EPISODES."mid" = MOVIES1."mid"
Where MOVIES."mid"=@id

That goes into Series.tv and my custom PVD field for grouping movies into series (e.g., Lord of the Rings, James Bond) goes into Series.movie. The applicable one then goes into my calculated custom Series. field using If(IsEqual([Media Sub Type],TV Show),[Name],[Series.movie][Series.tv]). I'm tagging series "master" records (dummy records for information about the whole series) with Media Sub Type="TV Show." This then sets Series. = Name to include the series name (i.e., title) for those items. Here's a screenshot (http://www.pix01.com/gallery/682885E4-EEF9-44DF-AA74-FC43A2882645/MC_Screenshots/bigpage.html?image=502608968_orig0.jpg).
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on May 02, 2009, 03:54:02 am

I don't do DVD, so I can only ask a dumb question... What would MC do if you changed the .ifo to .dvd;1 and associated that extension with MC? I expect it would not recognize it as a valid file type. But if by some miracle it does otherwise, then PVD won't object to .dvd;1 being set as a video file type. Otherwise, all I can say is "uh-oh." :-\

Nope, I also tried to set the file name in PVD to end with .dvd;1 but that did not work.

peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 05, 2009, 02:31:50 am
Back in the land of the living!

I just tried updating 500 movies with 'genre' data from PVD. I deleted some of the other fields from the plugin to simplify things a bit. A few other fields were not deleted, and these had no data present in the PVD library. Test worked fine, importing Genre while returning nothing for the other fields.

I selected the movie files, used Send To > PVDImport and almost immediately, I was getting exception errors in PvdImportContextMenu.exe and "application failed to initialise" errors in dwwin.exe:

(http://www.pix01.com/gallery/7E547B99-5004-478D-A4AC-37AD672F51A4/PVD_Crash/10346364870.jpg)
Click to enlarge. (http://www.pix01.com/gallery/7E547B99-5004-478D-A4AC-37AD672F51A4/PVD_Crash/1034636487_orig0.jpg)

(http://www.pix01.com/gallery/7E547B99-5004-478D-A4AC-37AD672F51A4/PVD_Crash/10346364871.jpg)
Click to enlarge. (http://www.pix01.com/gallery/7E547B99-5004-478D-A4AC-37AD672F51A4/PVD_Crash/1034636487_orig1.jpg)

Despite these errors, the plugin seemed to be importing the Genre data okay. After a minute or two (and lots of windows error soundclips :)) MC froze. I waited another few minutes, terminated the program and then reopened. The plugin had correctly imported genre data for 227 of the files. Given time it may have completed all of them - but the hundreds of error popups were disconcerting :)

I'll try with smaller batches of files and see if I can narrow down common elements to reproduce the errors. Maybe fields which are empty in PVD? Has anyone else seen this?

Just an aside... is there (or will there be) any way to simply 'disable' import of a field into MC without actually deleting it from the plugin menu?

Nope, I also tried to set the file name in PVD to end with .dvd;1 but that did not work.

This will definitely be a problem. MC handles DVDs quite differently, in that the "Filename" field in MC is more of a instruction of where to find the video files rather than the physical location of the file.
I only have a handful of DVD files and have been able to import the data correctly using plain text export (to MPL) from PVD. The database just has to refer to the *.dvd;1 extension. The fact this works with plain text export and not with the plugin... maybe has something to do with handling of the semicolon ";" ?
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on May 05, 2009, 02:36:47 am
If this is the bug I reported, it shouldn't be stopping you from running PVD and PvdImport simultaneously. If PVD is "stuck" running Firebird Embedded, edit your pvdconfig.ini so the server connection will be used. Mine looks like this:

Code: [Select]
[DataBase]
FileName=rix:C:\Data\PVD\MY MOVIES BETA 5.PVD
Mode=0
RecentFiles=..\..\Data\PVD\Dummy.PVD|rix:C:\Data\PVD\TEST.PVD
Network=1
RecentTypes=0;1;


Dumb question...I also would like to have mc and pvd open at once but I can't seem to find pvdconfig.ini...what am I missing?

peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 05, 2009, 03:44:29 am
I'll try with smaller batches of files and see if I can narrow down common elements to reproduce the errors. Maybe fields which are empty in PVD? Has anyone else seen this?

It seems to be the same problem I encountered—the system is overwhelmed with 500 instances of the program being launched at once. Using a process explorer, you can see what's going on—until that crashes too. :P I imagine there's going to have to be a way to "feed" the Sent To's at a rate the server can handle (most of) them.

Quote
Just an aside... is there (or will there be) any way to simply 'disable' import of a field into MC without actually deleting it from the plugin menu?

Good idea—that would be more convenient than using multiple configuration files.

Quote
This will definitely be a problem. MC handles DVDs quite differently, in that the "Filename" field in MC is more of a instruction of where to find the video files rather than the physical location of the file.

What if the .DVD;1 file was put in a playlist file?

I also would like to have mc and pvd open at once but I can't seem to find pvdconfig.ini...what am I missing?

The configuration file is saved in the application data directory. In XP that's C:\Documents and Settings\%USERNAME%\Application Data\Personal Video Database\. In Vista, I believe it's: C:\Users\%USERNAME%\AppData\Personal Video Database\. If you can't find the application data folder, it's probably because it's hidden.
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on May 05, 2009, 04:57:51 am


The configuration file is saved in the application data directory. In XP that's C:\Documents and Settings\%USERNAME%\Application Data\Personal Video Database\. In Vista, I believe it's: C:\Users\%USERNAME%\AppData\Personal Video Database\. If you can't find the application data folder, it's probably because it's hidden.

Thanks!...it now asks for a username and password...I did try my windows user password, the default firebird password and I created a user in pvd but none of them worked...any suggestions what it is looking for?

peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 05, 2009, 11:28:06 am
It seems to be the same problem I encountered—the system is overwhelmed with 500 instances of the program being launched at once.

Strangely enough, I don't get an exception for that many files.

I'd been hoping that JRiver would expose access to menus from the plugin environment.

The interprocess communication between the plugin and PvdImportContextMenu.exe is really a pain. I had to use named pipes since both Udp communication and Remoting wake up windows security. In addition to that, of course, framework 2.0 only supports named pipes via unmanaged calls, so there is a lot of overhead.

For now, we'll have to do batches of 50 and then I'll maybe look further into forcing one instance of the application + multiple threads when I get the motivation.
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 05, 2009, 05:19:11 pm
Quote
Strangely enough, I don't get an exception for that many files.

I got the impression it was the out-of-memory errors that triggered the rest. If this is true, the amount of memory you have would have a direct bearing on this. I have 512 Mb, and it happens (for sure) at 150 items. Maybe darichman has 2 Gb, and it happens at 500—and you have more than 2 Gb, so it doesn't?

Quote
The interprocess communication between the plugin and PvdImportContextMenu.exe is really a pain.

This stuff is way over my head, but if I understand correctly, you don't have any control whatsoever over the Send To function, so you can't even place an arbitrary limit on the number of files updated at one time. Maybe you should post a separate request for help. Maybe there's someone out there who has a solution, but is not following this topic. Failing that, your plugin is potentially important enough to MC users generally that we might hope J River development staff may offer some advice. Given the nature of the problem (it can crash Windows and corrupt the MC database), I don't think you can "release" the plugin until this is resolved.
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 05, 2009, 05:29:25 pm
Thanks!...it now asks for a username and password...I did try my windows user password, the default firebird password and I created a user in pvd but none of them worked...any suggestions what it is looking for?

What is asking for a username and password? Could this be a Vista thing—because you are changing a configuration in a protected folder? What happens if you copy the configuration file to the installation directory, and then run PVD with the -portable switch (that forces it to use the configuration file located there)?
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on May 06, 2009, 12:30:32 am
What is asking for a username and password? Could this be a Vista thing—because you are changing a configuration in a protected folder? What happens if you copy the configuration file to the installation directory, and then run PVD with the -portable switch (that forces it to use the configuration file located there)?

Pvd is asking for a username and password when I change the ini to connect to the database using the network method.

peter

Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 06, 2009, 01:29:19 am
Pvd is asking for a username and password when I change the ini to connect to the database using the network method.

Sorry, I haven't seen that one before. I suggest you install 0.9.9.6 and see if that makes a difference—the connect to server problem has been fixed. If that doesn't work, ask for help in the the PVD forum.

But first, maybe try this query—to see if any users/passwords actually exist:

Code: [Select]
Select
  USERS."name",
  PERMISSIONS."password"
From
  USERS Inner Join
  PERMISSIONS On USERS."uid" = PERMISSIONS."uid"

It seems the security, if that's what you've run into, is not very secure. ;)
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on May 06, 2009, 02:03:30 am
Thanks Rick got it working by upgrading to 0.9.9.6  :)

Uhm better not connect that database to the internet  :o ;)

peter

Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 06, 2009, 04:07:27 am
Did any of you guys test the new coverart functionality and ignore fields of 1.0.5?
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on May 06, 2009, 04:14:02 am
Did any of you guys test the new coverart functionality and ignore fields of 1.0.5?

Oops I did not see there was a new version...i'll try it later today

peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 06, 2009, 11:01:18 am
1.0.6
-----
o Fixed by removing plugin skinning: Critical parts of the plugin are not visible in the ThunderStorm or Blue Steel skins.
o Fixed: An empty MC field leads to an exception

1.0.5
-----
o Cover art Functionality. "Emulates" the setup of the tools/options dialog in MC -- There is no API accessible in plugins for this).
o New Ignore field: Selected fields items will not be updated
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on May 06, 2009, 11:28:29 am
1. Ignore option works for me.

2. Coverart > filepath\folder.jpg did not work. I'll try it again tomorrow.

peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 06, 2009, 02:34:43 pm
2. Coverart > filepath\folder.jpg did not work. I'll try it again tomorrow.

Which value in the filename tag do you see?
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on May 06, 2009, 03:27:19 pm
Which value in the filename tag do you see?

In mc? filename field = J:\Film HD Gezien\Requiem for a Dream\Requiem for a Dream 1080p.mkv

peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 06, 2009, 04:06:02 pm
oops, should have been, "what's the value of [Image File]", sorry...
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 06, 2009, 04:11:52 pm
o Cover art Functionality

Fabulous! I've always kept a poster beside my video files, and used PVD to maintain this. MC, of course, has no difficulty finding and using these. So I hadn't even thought of this as being necessary. But now the plugin takes care of this automatically—particularly useful now that I have PVD automatically grabbing screenshots and using one as a poster for episodes. Now I automatically get a unique poster for each new episode!

Well, not quite automatically—I still have to run the plugin. It would very cool if it could recognize new files and do an update automatically. Don't tell me—plugins don't have access to Auto Import functionality. ::)

Quote
New Ignore field: Selected fields items will not be updated

It works. I'm sure there are reasons to not update a field that I'm unable to think of right now, but I do see this as a convenient way to maintain alternate expressions for one field. I suppose I should just try it, but what happens if there two expressions active for one field (i.e., I forget to set one to "ignore")?

Quote
Fixed by removing plugin skinning: Critical parts of the plugin are not visible in the ThunderStorm or Blue Steel skins.

It looks so pretty in my ThunderStorm skin! :D

Quote
Known Issues 1.0.6:
o Plugin doesn't handle DVDs (.ifo, .vob, etc.)

Has anyone tried my suggestion of putting them in a playlist? (I'd try it myself, but I don't do DVD's.)

Raldo, keep up the good work! This is really starting to look like an awesome solution. 8)
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 06, 2009, 05:33:54 pm
Any ideas how to get this working with ripped dvd's? With mc you cant just point pvd to the .ifo file in the VIDEO_TS folder because mc creates its unique filename in the filename field (ie .dvd;1) Actually I would prefer if I could just use .ifo with mc, but I have not seen an option in mc to do just that.

peter
Does PVD know how to import dvd (ts, ifo, vob)? I've tried but PVD seems to ignore the folder...
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 06, 2009, 06:25:52 pm
PvdImport v1.0.6
Personal Video Database v0.9.9.6

Works well with the new version of PVD - just ran a batch of 100 files
The new ignore field is working for me, both in Test mode and "Send to >" mode. Rick, I guess this is also useful in testing (eg I might wish to test the plugin on my main library of files, but don't wish all my fields to be overwritten :))
Plugin displays correctly in all stock MC skins

Issues:
Assign coverart isn't working for me - Does the name of the image file have to match that of the video file? I store my coverart in the same folder as the video file, and set the plugin to "InTheSameFolderAsTheFile". There is only one video file and one image file per folder. The name of the image file does not always match the name of the video file (this might be the problem?). One example is

Special Characters not supported? This fails to update:

The columns are too narrow to see the column header, so the user might not know what the checkboxes are for...

The plugin is still updating MC fields in test mode

Suggestions:
Would you consider giving the user the option to specify whether or not they wish to overwrite existing values in MC fields?
A progress bar and status report at the end might be nice somewhere down the track :) The plugin does not seem to update files in any particular order (at least as far as the user can tell by watching the fields get populated in a file list view within MC)

Quote
Has anyone tried my suggestion of putting them in a playlist? (I'd try it myself, but I don't do DVD's.)

What do you mean by putting them in a playlist, Rick? You can successfully import data using an MPL file exported from PVD - Is this what you mean?

This plugin has come a long way very quickly... it's at the point where it's already quite usable :D
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 06, 2009, 06:32:22 pm
Does PVD know how to import dvd (ts, ifo, vob)? I've tried but PVD seems to ignore the folder...

I only have 2 DVDs in my MC library - I copy from MC into an excel spreadsheet and import them into MC that way. At least this way both MC and PVD refer to the DVD movie in the same way.
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on May 07, 2009, 12:28:59 am
Does PVD know how to import dvd (ts, ifo, vob)? I've tried but PVD seems to ignore the folder...

I point PVD to the .ifo file. If you could match it with the pseudo filename in mc then we could make the match.

peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on May 07, 2009, 02:46:19 am
oops, should have been, "what's the value of [Image File]", sorry...

ok, it says [image File] = Folder.jpg but there is no Folder.jpg in the filepath.

peter

Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 07, 2009, 04:00:13 am
What do you mean by putting them in a playlist, Rick?

I was suggesting a straw to grasp at for getting around the "DVD;1" problem. I've since realized I actually do have a few of these (not previously imported to MC) that I could experiment with. The playlist idea is clearly a dead end, but I did discover some weird and wonderful things. This is what I did:

1. Imported DVD rips to MC. Noted the magical DVD;1 extension, but File Type was still IFO. So...

2. Changed extension in MC (which changes actual extension in file system) to IFO. (To be clear, the complete filename in each case is VIDEO_TS.IFO. I don't understand these things, but I guess the IFO file contains information about the VOB files in the directory.) Selected Play in PVD, and each played fine in MC—the associated media player. They also, of course, played fine when played directly in MC. So far so good, so...

3. Ran PvdImport. The meta information is not updated. What? Perhaps the smart-ass MC won't recognize them as valid video files. So...

4. Changed the extensions (in both PVD and MC) to LOG. (Relax—that's the extension I set up for my dummy files. To my MC, they're text files which included in the video view. The effect would be the same if I changed them both to AVI.) You guessed it—now PvdImport works fine.

5. Changed the extensions back to IFO. Now here's the weird part...

6. Changed some information in PVD, and tried importing again. PvdImport now works fine on the IFO files (not new ones, just the ones I did the above to).

So the plugin will handle DVD files, but the workaround is not practical for anyone who has significant numbers of them. Also, there must be a reason why the DVD;1 scheme is used—I have no idea what adverse consequences there might be in not complying with it. :-\

I only have 2 DVDs in my MC library - I copy from MC into an excel spreadsheet and import them into MC that way. At least this way both MC and PVD refer to the DVD movie in the same way.

Okay, now I understand what you're doing. This is another way to circumvent MC application of the DVD;1 "extension" to the files! Still an awkward workaround, and too late if the files have already been imported. If we could just reduce the solution to simply changing the file extension...
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 07, 2009, 04:30:55 am
I point PVD to the .ifo file. If you could match it with the pseudo filename in mc then we could make the match.

What does "I point PVD to the .ifo file" mean? can you import single files? I haven' found that feature yet.

And, yes, The way PvdImport works *now* is that [File Name] must have an exact match with one of the filenames listed between pipe symbols "|" in PVD.

But as I wrote above, PVD does't import my .ifo (or whatever) files.
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on May 07, 2009, 04:31:44 am
Also, there must be a reason why the DVD;1 scheme is used—I have no idea what adverse consequences there might be in not complying with it. :-\

My guess is that they wanted to present a dvd as one entity, because for "normal" users the correct .ifo file might not be so obvious. I can see no other reason for not using the .ifo directly, no other app I am aware of creates this kind of dummy extension to refer to a dvd.

peter

Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on May 07, 2009, 04:36:53 am
ted between pipe symbols "|" in PVD.

But as I wrote above, PVD does't import my .ifo (or whatever) files.

It does for me:

movies > new movie master > File select the ifo file > go
At the end it might ask for files to include...flag video_ts.ifo

You will end up with something like this for file path in pvd:
J:\Film DVD Gezien\Bringing up Baby\VIDEO_TS\VIDEO_TS.IFO

peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 07, 2009, 04:42:02 am
You will end up with something like this for file path in pvd:
Thanks, that's just what I needed. I'll see what I can do wrt. special handling of DVDs...
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on May 07, 2009, 04:44:47 am
Actually when I use set in the plugin settings to point to the ifo file and then use test it will show the correct sqlresults. The only problem it does not match with mc's filename. Maybe you could try to make it so that if the file has an ifo extension then try a match with mc's ,dvd;1 extension.

peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 07, 2009, 04:48:18 am
Actually when I use set in the plugin settings to point to the ifo file and then use test it will show the correct sqlresults. The only problem it does not match with mc's filename. Maybe you could try to make it so that if the file has an ifo extension then try a match with mc's ,dvd;1 extension.
Yes, something in that direction.
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 07, 2009, 05:26:53 am
Quote
Yes, something in that direction.

I knew you'd see the light if we came up with an ugly enough workaround.  ;D

Quote
What does "I point PVD to the .ifo file" mean? can you import single files? I haven' found that feature yet.

Just to clarify... the path can be entered "manually" in the field, which is most commonly how this would be done. But the file scanner could also be configured to pick out the VIDEO_TS.IFO file, and only that IFO file. I point this out because I don't think the default configuration of the scanner does this. My DVD rips originally had all the IFO and VOB files in the path. That seemed to work okay, but it would prevent the plugin from matching the files.

Anyway, hopefully it's safe to say...

1. The PVD file scanner can be configured to use VIDEO_TS.IFO as the file path.

2. This will work fine for playing the video.

3. The Plugin can be modified to match PVD's IFO's to MC's DVD;1's.
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 07, 2009, 05:35:07 am
3. The Plugin can be modified to match PVD's IFO's to MC's DVD;1's.
Exactly.
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 09, 2009, 08:29:03 am
1.0.7
-----
o Fixed: Plugin doesn't handle DVDs (.ifo, .vob, etc.)
o Fixed: folder.jpg for cover art doesn't work properly.
o Fixed: If a movie filename and or path contains a special character like ' or " (umlaut) then the data will not be imported from pvd.

I haven't been able to test DVD handling properly since PVD is acting up on me (I cannot import new files). But the fix in this release handles the notorious .dvd;1 (mc) vs. .ifo (PVD). So if your file is called .ifo in PVD, it should work (?).

Please test for special characters. There are a lot of levels in the plugin where this can go wrong, so I'd apprerciate feedback!
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 09, 2009, 08:44:33 am
Issues:
Assign coverart isn't working for me - Does the name of the image file have to match that of the video file? I store my coverart in the same folder as the video file, and set the plugin to "InTheSameFolderAsTheFile". There is only one video file and one image file per folder. The name of the image file does not always match the name of the video file (this might be the problem?). One example is
  • Video:  F:\Film\A Beautiful Mind\A.Beautiful.Mind.2001.DVDRip.x264.AC3.mkv
  • Image:  F:\Film\A Beautiful Mind\A Beautiful Mind.jpeg
How is it not working? I believe that there is no "rule" in MC on how this relation ship should work for this coverart selection. Currently, I generate the jpg name with the [name] tag of the file. Are you saying MC doesn't display cover art? Is the [image file] tag correct?

The columns are too narrow to see the column header, so the user might not know what the checkboxes are for...
Yes, I still haven't figured out how to fix the tooltip for each column. It's a space saving issue...

The plugin is still updating MC fields in test mode
Well, updating the fields is currently part of the test. Maybe some tick box to deselect update?

Suggestions:
Could you do me a favor and start a new suggestions thread?
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on May 09, 2009, 09:22:20 am
1.0.7
-----
o Fixed: Plugin doesn't handle DVDs (.ifo, .vob, etc.)
o Fixed: folder.jpg for cover art doesn't work properly.
o Fixed: If a movie filename and or path contains a special character like ' or " (umlaut) then the data will not be imported from pvd.

I haven't been able to test DVD handling properly since PVD is acting up on me (I cannot import new files). But the fix in this release handles the notorious .dvd;1 (mc) vs. .ifo (PVD). So if your file is called .ifo in PVD, it should work (?).

Please test for special characters. There are a lot of levels in the plugin where this can go wrong, so I'd apprerciate feedback!


1. Dvd import. I just tried a batch of 50 and 100% success!

2. Special characters. Reimported the movies that failed before...all ok now!

3. Folder.jpg also worked!

Big thanks for your hard work! :)

peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 09, 2009, 09:39:53 am
1. Dvd import. I just tried a batch of 50 and 100% success!

2. Special characters. Reimported the movies that failed before...all ok now!

3. Folder.jpg also worked!

Big thanks for your hard work! :)

peter
You're welcome. Thanks for testing!
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 09, 2009, 05:05:15 pm
There are a lot of levels in the plugin where this can go wrong, so I'd appreciate feedback!

It seems fine to me. Amélie and Léon are happy now. ;)

Quote
I haven't been able to test DVD handling properly...

You should still be able to enter the file path in PVD manually—in order to test. That might also help to isolate the problem. For example, if you were to do that and then Re-read video file information, what happens? Maybe the problem has something to do with scanning with MediaInfo, or maybe something unrelated to scanning.

Comments on top message instructions

2. Should be updated to refer and link to PVD 0.9.9.6.

3.1. Reference to connection problems should be removed.

4.1. Add a note that Test will update the test record in the database (surely no one will object as long as they know).

4.2 Add reference to the known issue regarding selecting too many files for update—for those who won't make it to the next message before attempting a mass update. ;)

Quote
Could you do me a favor and start a new suggestions thread?

That's a good idea, but it would be helpful to have an idea what you think should make into the first release version (1.1?) versus a later update (1.2?). For example...

1. Multi-threading (or whatever) to deal with the "too many files" issue.

2. Data conversion features (e.g., converting dates and ratings so they can be put in MC standard fields).

3. I don't think it's a big deal—and I guess this is the sort of thing the plugin doesn't have access to—but some sort of link between the "McFields" and the MC add/edit field dialog.

Personally, I think the first is one most important to resolve—if by only in some way preventing too many files being selected until a permanent resolution is implemented. Then would be a good time to get more users involved, and see what "bells & whistles" they are most interested in. Perhaps you should post a request for feature suggestions at the same time you post the release (i.e., "Please report problems here, feature requests to...").
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 10, 2009, 06:24:11 am
I did some performance testing :)

100 files ~ 30s
200 files ~ 90s
300 files ~ lots and lots of PvdImportContextMenu.exe errors!!

It appears I can do 200 files at a time consistently (2Gb ram)

How is it not working? I believe that there is no "rule" in MC on how this relation ship should work for this coverart selection. Currently, I generate the jpg name with the [name] tag of the file. Are you saying MC doesn't display cover art? Is the [image file] tag correct?

I totally missed the point here! I thought the plugin was meant to be updating coverart FROM an image already in the folder specified (eg. Folder.jpeg), but now I realise you've gone one step further and are actually pulling the coverart from PVD. Sorry, I'm a bit obtuse sometimes haha.

Quote
o Fixed: Plugin doesn't handle DVDs (.ifo, .vob, etc.)

This is working really well if PVD points to the *.ifo file

An observation, however - if PVD refers to a *.ifo file and MC refers to a *.dvd;1 extension, the plugin works flawlessly.
BUT If both PVD and MC point to *.dvd;1 the plugin fails.

I kept my PVD database pointing to *dvd;1 as this way MPL export worked okay - I guess that's no longer required though, so I may as well swap them over to *.ifo.

Quote
o Fixed: If a movie filename and or path contains a special character like ' or " (umlaut) then the data will not be imported from pvd.

I had a few of these - all good now :)

Could you do me a favor and start a new suggestions thread?

Sure thing, will read over this thread and start one soon.

2. Data conversion features (e.g., converting dates and ratings so they can be put in MC standard fields).

This would be nice. How does PVD store date data?
MC system is detailed here: http://yabb.jriver.com/interact/index.php?topic=50750.0 (http://yabb.jriver.com/interact/index.php?topic=50750.0)
and here: http://yabb.jriver.com/interact/index.php?topic=49173.0 (http://yabb.jriver.com/interact/index.php?topic=49173.0)

You're probably close to a release build now - I agree with Rick, in that the main limiting factor at the moment is the need to run small batches. I guess you can either roll the processes out sequentially as each one finishes, or alternatively contain the lookups in batches of 50 or so at a time (ie execute the first 50, wait til they're finished and then execute the second 50 etc etc.)

Loving the plugin so far, Raldo. Your hours of hard work will save us many more ;)
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 10, 2009, 03:51:39 pm
Quote
100 files ~ 30s

On my machine (512 Mb RAM) 100 files takes about 15 minutes. I probably shouldn't be wasting my time on anything other than shopping for a new machine, but it might be of some use to ponder why this is so. The number and complexity of the queries has some impact, but not much. Similarly, disabling my virus scanner and other processes. Any suggestions of other things I should try? PVD is rather sluggish, so I'm not expecting a miracle, but 10 seconds per file is so slow there must be something else going on.

Quote
This would be nice. How does PVD store date data?

Strings—at least that's the form of the query result. The format is MM/DD/YYYY 12:00 AM, so it doesn't sort properly in MC. Furthermore, even if they where available in a compatible decimal form, MC doesn't offer a date-type field. The parallel situation is the same with PVD—dates imported or created with a script have to go into a string field because there is no date-type for custom fields. Even if there were a SQL function for converting the data (I couldn't find one) so it could be stored in a standard data field, this wouldn't be a complete solution. I'm not sure what MC might otherwise do to it's date fields. For example, if the Date modified field used to store the equivalent from PVD, will MC overwrite that when it detects the a different time stamp on the media file? Regardless, it would be nice to be able to import custom dates (doing whatever conversion is necessary), but maybe it's not feasible.

What I'm doing now is importing the date strings to custom fields, and then using an expression column to reformat them as YYYY-MM-DD so they sort properly.

And then there's my custom composite rating field... ;D

Code: [Select]
=If(IsEqual(Math(4-IsEmpty([Rating.IMDb],1)-IsEmpty([Rating.Other],1)-IsEmpty([Rating.User],1)-IsEmpty([Rating.AllMovie],1)),0),0, Math(Math(If(IsEmpty([Rating.IMDb]),0,Field(Rating.IMDb)) + If(IsEmpty([Rating.Other]),0,Field(Rating.Other)) + If(IsEmpty([Rating.User]),0,Field(Rating.User)*2) + If(IsEmpty([Rating.AllMovie]),0,Field(Rating.AllMovie))) // Math(5-IsEmpty([Rating.IMDb],1)-IsEmpty([Rating.Other],1)-IsEmpty([Rating.User],1)*2-IsEmpty([Rating.AllMovie],1))*.5+.5))
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 10, 2009, 10:48:49 pm
Rick... how on Earth do you keep any tenure on your sanity when you have to wait that long each time! Your patience has earned you my undying respect :D

As far as dates go, there may not be any way of creating new date fields, but we could probably put our heads together and get release date from PVD into the standard [Date] field in MC.
MC stores date internally as  number of days since December 31, 1899 (time is stored in fractions of days). This system is used in Excel and some other MS programs.

If we could parse the date string from PVD into days, months & years it might be possible to calculate date in MC's internal date format. Is it worth the effort though? We can get year across easily - does having the month and day really make that much difference? hmmm....

Rick, your super composite rating field scares me. Just a little bit.
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 11, 2009, 04:09:21 am
Quote
does having the month and day really make that much difference?

No. For the release date, I'm happy with just the year. But there's date acquired, updated (i.e., meta data downloaded), viewed, modified (i.e., PVD record changed). If it's not possible to create a custom date field, I don't think there's any choice but to use strings. It would be nice to be able to put it in the YYYY-MM-DD format, however—to eliminate the need for expressions.

Quote
Rick, your super composite rating field scares me. Just a little bit.

I have to do something while waiting for imports to finish. Besides, it anyone asks, "Why does that get four stars?" I have a geeky answer for them. 8)
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 11, 2009, 05:45:36 am
No. For the release date, I'm happy with just the year. But there's date acquired, updated (i.e., meta data downloaded), viewed, modified (i.e., PVD record changed). If it's not possible to create a custom date field, I don't think there's any choice but to use strings. It would be nice to be able to put it in the YYYY-MM-DD format, however—to eliminate the need for expressions.

I have to do something while waiting for imports to finish. Besides, it anyone asks, "Why does that get four stars?" I have a geeky answer for them. 8)

You can use SQL to format your dates or manipulate the output from sql in various ways. Combine Format() with a stored procedure, and you can do practically anything you want with time, for example. I'm sure that you could produce an MC style date where the number of seconds (and fractions) are output from SQL.

 
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 11, 2009, 01:58:41 pm
Quote
I'm sure that you could produce an MC style date where the number of seconds (and fractions) are output from SQL.

Could you provide an illustration, for example, for saving PVD moddate to MC Date? Without documentation, I haven't a clue how to create a stored procedure or even how to use format() (which might be useful for a variety of things).

I still don't see much point in this if the only date field available to us in MC is Date. Am I missing something?
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 13, 2009, 03:57:19 am
[...] Without documentation, I haven't a clue how to create a stored procedure or even how to use format() (which might be useful for a variety of things).

Try googling; Firebird documentation is located online. "Firebird Format time date select".

I still don't see much point in this if the only date field available to us in MC is Date. Am I missing something?

True, I just wanted to point out that there are methods in SQL which can format the output.
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 15, 2009, 05:59:33 am
Code: [Select]
Select PEOPLE."name" || '\' || CAREER."role"
From PEOPLE Inner join CAREER on (CAREER."pid" = PEOPLE."pid")
Where CAREER."ctype" = 0 And CAREER."mid" = @id

This one gives you nested panes with actors and roles! So you can expand an actor within a pane in MC and see the different roles they've played underneath :)

Code: [Select]
Select CUSTOM_VALUES_LONGSTR."value"
From MOVIES
Inner join CUSTOM_VALUES_LONGSTR on MOVIES."mid" = CUSTOM_VALUES_LONGSTR."mid"
Inner join CUSTOM_FIELDS_MOVIES on CUSTOM_FIELDS_MOVIES."cmfid" =  CUSTOM_VALUES_LONGSTR."cmfid"
Where MOVIES."mid"=@id and CUSTOM_FIELDS_MOVIES."dispname" = 'Review'

Thanks for that one Rick.

So guys, any chance of a quick rundown on where to start with Firebird Maestro with the PVD database? I don't even know how to get it to read the database!
I have a bunch of custom short text fields I'd like to try out with the plugin...

Raldo, is there anything else specific you'd like us to test out?
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 15, 2009, 06:52:18 am
Code: [Select]
Select PEOPLE."name" || '\' || CAREER."role"
From PEOPLE Inner join CAREER on (CAREER."pid" = PEOPLE."pid")
Where CAREER."ctype" = 0 And CAREER."mid" = @id

This one gives you nested panes with actors and roles! So you can expand an actor within a pane in MC and see the different roles they've played underneath :)

Code: [Select]
Select CUSTOM_VALUES_LONGSTR."value"
From MOVIES
Inner join CUSTOM_VALUES_LONGSTR on MOVIES."mid" = CUSTOM_VALUES_LONGSTR."mid"
Inner join CUSTOM_FIELDS_MOVIES on CUSTOM_FIELDS_MOVIES."cmfid" =  CUSTOM_VALUES_LONGSTR."cmfid"
Where MOVIES."mid"=@id and CUSTOM_FIELDS_MOVIES."dispname" = 'Review'

Thanks for that one Rick.

So guys, any chance of a quick rundown on where to start with Firebird Maestro with the PVD database? I don't even know how to get it to read the database!
I have a bunch of custom short text fields I'd like to try out with the plugin...

Raldo, is there anything else specific you'd like us to test out?

I'd use some other client/tool than Maestro. My trial ended yesterday, so I'll find some other free tool. I think there was some discussion on this in the PVD forums.

I'll be releasing a new version soon; I'll add some progress indicator there pluss a method to update all fields.

The reason for the latter feature is that I've spent a lot of time trying to get "Send To External" to work better. I'm still using Named Pipes after a tiresome digression into mailslots.

I.e. I'm giving up on mass updates via "Send to", hoping that JRiver will expose parts of the menu  system to plugins...
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on May 15, 2009, 07:52:07 am

I.e. I'm giving up on mass updates via "Send to", hoping that JRiver will expose parts of the menu  system to plugins...

I hope so this is a great opportunity! In the future I would love to just say update all my movies info in jrmc from the pvd database...

peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 15, 2009, 07:57:43 am
aTagger (a plugin no longer in circulation) updated music files by downloading information from AMG and directly updating MC fields. While the first part is obviously different in PVDImport, maybe we can learn something from how it updated the files in MC? The plugin allowed the user to specify a playlist (or playing now) and updated any files in that playlist - all the user had to do was execute the plugin from the actual plugin page (progress & status was shown here too). It worked really well... there was no "Send To" component to it at all...

If you could emulated this, all the user would need to do would be to drag all their movie files into a playlist (or even create a smartlist).
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 15, 2009, 09:40:10 am
aTagger (a plugin no longer in circulation) updated music files by downloading information from AMG and directly updating MC fields. While the first part is obviously different in PVDImport, maybe we can learn something from how it updated the files in MC? The plugin allowed the user to specify a playlist (or playing now) and updated any files in that playlist - all the user had to do was execute the plugin from the actual plugin page (progress & status was shown here too). It worked really well... there was no "Send To" component to it at all...

If you could emulated this, all the user would need to do would be to drag all their movie files into a playlist (or even create a smartlist).

Ok, let me clarify: "Send To" will remain. I'm just saying that I've given up the effort to do it for more than, say, 50 files.

I'm testing a mass import feature, done from the plugin as you're describing in your post. The mass import will be configured via a MC search expression. You can set this expression anyway you like; In my case, I have set it up like this: [Media Type]=[Data],[TV],[Video] , but if you prefer a playlist, you can set it up like this: playlistid==902071051,=491747710
Title: Re: Personal Video Database Import plugin - Test log
Post by: )p( on May 15, 2009, 09:59:04 am
Wow sounds the future is coming much sooner then I expected  :)  ;)


peter
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 15, 2009, 08:57:54 pm
This one gives you nested panes with actors and roles!

I was putting Select CAREER."role" || ' (' || PEOPLE."name" || ')' into Roles (as a compliment to Actors). Why didn't I think of this?! So close, yet so far. ::) 

I like this much better—thanks! ;)

I'd use some other client/tool than Maestro. My trial ended yesterday, so I'll find some other free tool. I think there was some discussion on this in the PVD forums.

I use Flyspeed SQL Query (http://www.activedbsoft.com/overview-querytool.html) (free version). Although I'm not in a position to judge such things, the fact I'm able to use it suggests it's suitable to the purpose.

Quote
I'm testing a mass import feature, done from the plugin as you're describing in your post.

I've been unable to resolve my speed issue (i.e., the update taking ~10 seconds/record)—by eliminating other processes or changing the server configuration. So I'm looking forward to this as a possible solution. The "Send to" method will still be handy for updating a handful of new items.
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 16, 2009, 05:46:00 am

1.0.8:
----------
o feature: Mass import, uses MC expressions. See User manual for details.
o feature: Write only empty fields
o Improved: Thread handling when importing many files.
o Feature: Progress bar and current import status text
o Improved: Some changes in database handling (open/close)
o Improved: Internal code structure

There are new fields in the PvdImportConfiguration.xml file, so if you want to see the default installation values, you must remove the old file before you do a reinstall.

This release contains many changes, so please test thoroughly!

*********************
Known Issues 1.0.8:
*********************
o Don't push test button until sql results are ready, the application will hang
o Tooltip for field configuration headers doesn't show properly.
o If you select too many files (in the hundreds) in "Send To...", the PvdImportContextMenu.exe applications may get confused. Select fewer (50, maybe)!
o There should be a cancel button for mass imports.
o Some drawing issues when resizing MC.
o User information (lower window) is not updated properly during import.


Title: Re: Personal Video Database Import plugin - Test log
Post by: Two Wire on May 16, 2009, 06:08:36 pm
New user here and need some help.

1. Clicking on "Connect to Server" in PVD opens a dialog box requesting a Password and User name. I tried "sysdba" and "masterkey," but these did not work. (I got these from the Firebird Quick users guide.) Since I cannot get beyond this point, I doubt if I am connected. But, I do get this at the bottom of the Pvdimport plugin:

Failed to find Pvd database file file:
Trying default location: C:\Documents and Settings\Jerry\My Documents\Personal Video Database\movies.pvd
Success...
Plugin Initiated OK

2. Raldo's instructions refer to a "Start File " button. All I see is a "Start" button. Are these the same? Anyway, clicking on this button ask for opening a backup copy from the MC library. I selected a copy to open and had to leave the plugin to select a movie file. I did this, and returned back to the plugin to click on the "Test" button. As far as I could tell, nothing happened. Going back to the selected file, I did not see any additions, or changes, to the tags.

3. Additional information: Because I wanted to use an external player, I have been changing all of the file name extensions from ".DVD;1" to ".ifo"

4. Obviously I am doing something wrong and would like some help.

Thanks

Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 16, 2009, 11:14:10 pm
1.0.8 feature: Mass import, uses MC expressions. See User manual for details.

The use of an expression to select items is clever. Anyone who thinks this won't accommodate their workflow can be referred to one of the MC expression gurus. ;)

The mass import seems to work okay, although I see no change in the speed. So I suppose that issue has to do with my configuration of the Firebird server or, more likely, just a general lack of resources. At least now I can start a mass update and go to bed...

But there does seem to be a problem. The plugin makes the server behave as if it has a memory leak. It adds about 2,000 K "Private bytes" as each item is processed, and releases none—not as it is running, or after it has finished. The memory allocated is only released after MC is closed—so it seems clear the plugin is at fault. As it happened, the test batch was not big enough for me to see what might have happened if there were no more memory to be allocated.

Quote
Improved: Thread handling when importing many files.

I assume this refers to the handling of updates via Send To. I tried that on 100 items (which likely would have caused problems on my system before). Now I observed a flurry of 40 - 50 instances of PvdImportContextMenu.exe which quickly finished. The import then seems to run exactly the same as a mass import—with the plugin showing the progress and the server consuming memory.

This make me wonder why the Send To method is still used. If it didn't exist and I wanted to update a selection of (i.e., highlighted) files, I would right-click, send them to a playlist (say, "PvdImport"), and then specify p="PvdImport" for the import. If I then cleared the playlist, it would be available for a new batch. Could the plugin not be programmed to do, in effect, the same thing? Or is this one of the many things you should be able to do with a plugin, but can't?  :o
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 16, 2009, 11:49:09 pm
New user here and need some help.

1. Clicking on "Connect to Server" in PVD opens a dialog box requesting a Password and User name. I tried "sysdba" and "masterkey," but these did not work. (I got these from the Firebird Quick users guide.) Since I cannot get beyond this point, I doubt if I am connected. But, I do get this at the bottom of the Pvdimport plugin:

Which version of PVD are you using? Peter had this problem earlier in the thread, but said it went away when he updated to 0.9.9.6.

o feature: Mass import, uses MC expressions. See User manual for details.
o feature: Write only empty fields
o Improved: Thread handling when importing many files.
o Feature: Progress bar and current import status text

Just did an import with 560 files. Took about 4 minutes and had no issues whatsoever. You've really done an amazing job here :)

I haven't had the resource issues Rick mentioned above, but I don't don't really have a slower machine to test it on either :-\

I like the option to leave fields alone if they already have values.
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 17, 2009, 12:49:14 am
Quote
I haven't had the resource issues Rick mentioned above, but I don't really have a slower machine to test it on either

We could swap machines, if you like. ::)

Can you check a process explorer to see how much memory the Firebird server has used and whether it's released after an update? Maybe that's unique to my system as well, but I doubt it...
Title: Re: Personal Video Database Import plugin - Test log
Post by: Two Wire on May 17, 2009, 11:24:06 am
Thanks Darichiman for the reply.

I am using the latest version, pvd0996. I have tried this on three different computers and in each case after I enter the path to the database, a dialog box pops up requesting a username and password. I don't use any passwords on my personal computers, so this must be a request PVD is generating after firefox is installed. I tried deleting the config.ini file and this did not help.

Clicking cancle on the dialog box will give me an error code of 24. I can then manually open the database from the file menu, but of course, it will not be connected to the server.

This seems to be a problem with PVD, but that is as far as my troubleshooting skills can take me.

I like the looks of this plugin and it seems to have a lot of potential.
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 17, 2009, 12:27:59 pm
This seems to be a problem with PVD, but that is as far as my troubleshooting skills can take me.
Ok, it seems as if you have managed to get PVD up and running in "embedded Firebird mode".

From there, you can still get your DB up and running, just remember to close PVD and then restart MC. The reason for this is that PVD in embedded mode will lock down the PVD database, so that it cannot be accessed via the server.

Then, with a populated DB, PvdImport should be able to import data.
Title: Re: Personal Video Database Import plugin - Test log
Post by: Two Wire on May 18, 2009, 11:53:22 am
Thanks Roldo!  I have the plugin working, but maybe not completly. The "Send To" method is the one I have been expermenting with, and the "Year" and "IMDB Rating" fields are not being populated. They appear, however to be populated in PVD. I'm going to play with this this as it may be a problem on my end.

My main question, however, relates to the use of the "Set" and "Test" buttons.

Quote
o Press "set file"

I assume this is the "Set" button, and when I click on it, an "Open" window appears showing my
Movies.PVD database file. When I select the file and click on open, nothing appears to happen.


Quote
and select one of your video files that you know exists in the PVD database.

Should a list of the files in my Movies.PVD appear from which I can select the appropiate file? If I go back into Media Center and select a file, and then return to the plugin, won't I lose my selection of the file? That's what seems to happen.


Quote
o Press "test" (Warning: pressing "Test will update your library too)
                               

Nothing happens.

Quote
o The fields in the "SqlResult" should now be populated by data.


The "SqlResults are shown ony when I use the "Send To" method.


Finally, This is a supurb tool and will save me much time. Thanks for all the hours you have
spent in it's development. First, MC, then PVD, and now PVDIMPORT. How can it get any better






Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 18, 2009, 05:39:50 pm
Quote
I assume this is the "Set" button, and when I click on it, an "Open" window appears showing my
Movies.PVD database file. When I select the file and click on open, nothing appears to happen.

This is just for selecting the video file to be used for the test. All that should happen is the path of the file selected will be recorded in the box beside "Set."

Quote
Should a list of the files in my Movies.PVD appear from which I can select the appropiate file?

No, it's a file selection dialog for picking a file on your hard drive—pick one that you know is in your PVD database. Remember, it's the common media file that is the "link" between the two programs.

Quote
The "SqlResults are shown ony when I use the "Send To" method.

This suggests you have it configured correctly, but have not specified a valid test file. It may not refresh immediately, but "SqlResults" are shown for the file last processed.

Quote
the "Year" and "IMDB Rating" fields are not being populated.

Year needs to go into an integer-type field, which can then be displayed directly (i.e., choose it as a column in a view).

The "X.X" Imdbrating needs to go into a decimal-type field, which will be displayed as "X.XX". To display this as an MC-style star rating, an expression field is required. This screenshot (http://www.pix01.com/gallery/682885E4-EEF9-44DF-AA74-FC43A2882645/PvdImport_Screenshots/bigpage.html?image=281187350_orig0.jpg) illustrates how this should be set up. First, set data/edit-type to decimal/five stars. Then change it to "Calculated data" and enter the expression. Save, and then reopen the configuration for the new field. For reasons I don't understand, the data/edit-type reverts to the default. Change it back to decimal/five stars, and then reselect "Calculated data." Hopefully, the settings will then "stick."

Quote
Finally, This is a supurb tool and will save me much time. Thanks for all the hours you have
spent in it's development. First, MC, then PVD, and now PVDIMPORT. How can it get any better

+1 ;)
Title: Re: Personal Video Database Import plugin - Test log
Post by: Two Wire on May 18, 2009, 09:56:26 pm
Rick.ca, thanks for taking the time to clear this up for me. In hindsight, I should have figured this out. I just didn't drill down deep enough. Works great!    ;D

I will change the field formats for the "year" and "IMDB" fields as you suggest.

 
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 19, 2009, 02:08:07 am
The "SqlResults are shown ony when I use the "Send To" method.

Btw: If you read some of the earlier posts in this thread, you will find some tips on how to get the server connection of PVD up and running.
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 19, 2009, 12:48:24 pm
1.0.9:
----------
o feature: Mass import Stop Button
o feature: Removed Indexing of lists.
o Feature: Software User Manual (SUM) is now included in the installation. A button (SUM) invokes Notepad.exe which loads the file.
o Fixed: Some drawing issues when resizing MC.
o Fixed: Tooltip for field configuration headers doesn't show properly. You can deselect tooltips with a checkbox
o Fixed: Don't push test button until sql results are ready, the application will hang
o Fixed: User information (lower window) is not updated properly during import.

*********************
Known Issues 1.0.9:
*********************
o If you select too many files (in the hundreds) in "Send To...", the PvdImportContextMenu.exe applications may get confused. Select fewer (50, maybe)!

Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 19, 2009, 03:56:35 pm
1.0.10:
----------
o Fixed: Test filename would occasionally be truncated so that no data was imported

*********************
Known Issues 1.0.10:
*********************
o If you select too many files (in the hundreds) in "Send To...", the PvdImportContextMenu.exe applications may get confused. Select fewer (50, maybe)!
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 19, 2009, 05:25:35 pm
Quote
1.0.10: Fixed: Test filename would occasionally be truncated so that no data was imported

That's better. ;)

Quote
1.0.9: feature: Mass import Stop Button

Unlike many such things in Windows, it actually stops the process immediately! Handy for those of us with slow machines.

Quote
feature: Removed Indexing of lists.

Agreed. If I somehow lose the correct sort order, I can just re-import the record.

Quote
Feature: Software User Manual (SUM) is now included in the installation.

There's no such feature in either PVD or MC. Will users understand what it's for? ;D

Maybe the button should be labelled "Help."

Quote
Fixed: Tooltip for field configuration headers doesn't show properly. You can deselect tooltips with a checkbox

This helps a lot—removes ambiguity about what buttons do, etc. Being able to turn them off is also appreciated.

Quote
Fixed: User information (lower window) is not updated properly during import.

I see only "Plugin Initiated OK." Perhaps this is by design where there are no errors to report. But this would be more useful if it was more like an activity log—maybe including the last progress message of each import. It would give the user a useful summary of what has been done during the session—particularly the last item updated.

But there does seem to be a problem. The plugin makes the server behave as if it has a memory leak. It adds about 2,000 K "Private bytes" as each item is processed, and releases none—not as it is running, or after it has finished. The memory allocated is only released after MC is closed—so it seems clear the plugin is at fault. As it happened, the test batch was not big enough for me to see what might have happened if there were no more memory to be allocated.

I see no change in this behaviour. Any idea what's wrong?
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 20, 2009, 02:19:15 am
I see no change in this behaviour. Any idea what's wrong?

Nope, not yet.
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 20, 2009, 04:09:22 am
There's no such feature in either PVD or MC. Will users understand what it's for? ;D

Maybe the button should be labelled "Help."

Well it's not really just help either. The document explains some necessary steps you need to go through to get PvdImport up and running.

Unless some "better" word comes up which encompasses both "Help" and "Required Setup" I think "Software User Manual" is best for now. The tooltip expands on the abbreviation too.
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 20, 2009, 06:06:16 pm
1.0.11:
----------
o Fixed: Runtime information in lower Listbox. Added some logging information.
o Fixed: There is a memory leak in the Firebird server. This memory leak is related to PvdImport sql reads.

*********************
Known Issues 1.0.11:
*********************
o If you select too many files (in the hundreds) in "Send To...", the PvdImportContextMenu.exe applications may get confused. Select fewer (50, maybe)!

Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 20, 2009, 08:31:16 pm
Quote
Fixed: Runtime information in lower Listbox. Added some logging information.

Perfect! ...except it reports "Sucess!" when I'm looking for "Success!" ;)

Suggestions:

Quote
Fixed: There is a memory leak in the Firebird server.

Well done! Now it doesn't appear to use any additional memory at all. And as a bonus, the processing seems to be ~20% faster.



The plug seems about ready for prime time. Have you thought about how to "release" it? Or what more we need to do before it's ready? I imagine new users are going to demand help with the SQL statements. Maybe we should work on the default PvdImportFieldConfig.xml so it includes sample (i.e., Ignore="true") queries for each type of custom field, handling of dates and ratings, and various options (e.g., saving actors/roles as a nested field).
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 21, 2009, 06:40:41 am
Perfect! ...except it reports "Sucess!" when I'm looking for "Success!" ;)
:)

Suggestions:
  • Include a date/time stamp on each entry...
  • ...and I wonder if there should be an option for the recording of a date/time stamp in a custom field (e.g., if the user creates, say, a PvdImport.Updated field, the plugin will record it).
  • Add results to "log.txt" in PvdImport directory.


No more new stuff for now, let's stabilize and release...

The plug seems about ready for prime time. Have you thought about how to "release" it? Or what more we need to do before it's ready? I imagine new users are going to demand help with the SQL statements. Maybe we should work on the default PvdImportFieldConfig.xml so it includes sample (i.e., Ignore="true") queries for each type of custom field, handling of dates and ratings, and various options (e.g., saving actors/roles as a nested field).

I think the best thing to do is, as you suggest, to gather up some examples of fields, and include them in the default config file. Tag those examples as Ignore.

Maybe you and Darichman could cooperate on this?

In the meantime I'm going to do some internal "structural changes" which I've been wanting to do for some time.
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 22, 2009, 02:06:36 am
Quote
I think the best thing to do is, as you suggest, to gather up some examples of fields, and include them in the default config file. Tag those examples as Ignore.

Following is my shot at a default field configuration file. I've attempted to group the fields in a logical fashion, and have included comments to explain things. It's still hard to follow, so I've posted the configuration file and the Excel worksheet I used to create it here (http://www.videodb.info/forum_en/index.php?topic=1318.0).

Code: [Select]
<?xml version="1.0" encoding="utf-8"?>
<Organizer xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <!--Version: rick.ca 2009-05-21-->
  <!--______________-->
  <!--IDENTIFICATION-->
  <ConfigItem McField="Name" PvdExpression="Select MOVIES.&quot;title&quot; &#xD;&#xA;From MOVIES &#xD;&#xA;Where MOVIES.&quot;mid&quot;=@id" />
  <ConfigItem Ignore="true" McField="Original Title" PvdExpression="Select MOVIES.&quot;origtitle&quot; &#xD;&#xA;From MOVIES &#xD;&#xA;Where MOVIES.&quot;mid&quot;=@id" />
  <ConfigItem McField="Year." Type="Integer" PvdExpression="Select MOVIES.&quot;year&quot; &#xD;&#xA;From MOVIES&#xD;&#xA; Where MOVIES.&quot;mid&quot;=@id"></ConfigItem>
  <ConfigItem McField="Series" PvdExpression="Select MOVIES1.&quot;title&quot;&#xD;&#xA;From MOVIES &#xD;&#xA;Inner Join EPISODES On MOVIES.&quot;epid&quot; = EPISODES.&quot;epid&quot; &#xD;&#xA;Inner Join MOVIES MOVIES1 On EPISODES.&quot;mid&quot; = MOVIES1.&quot;mid&quot;&#xD;&#xA;Where MOVIES.&quot;mid&quot;=@id">
    <!--Get series name for episode record-->
  </ConfigItem>
  <ConfigItem McField="Season" PvdExpression="Select EPISODES.&quot;season&quot;&#xD;&#xA;From MOVIES Inner join EPISODES on MOVIES.&quot;epid&quot; = EPISODES.&quot;epid&quot;&#xD;&#xA;Where MOVIES.&quot;mid&quot;=@id">
    <!--Replacing SQL that returns index rather than number-->
  </ConfigItem>
  <ConfigItem McField="Episode" PvdExpression="Select EPISODES.&quot;epnum&quot;&#xD;&#xA;From MOVIES Inner join EPISODES on MOVIES.&quot;epid&quot; = EPISODES.&quot;epid&quot;&#xD;&#xA;Where MOVIES.&quot;mid&quot;=@id">
    <!--Replacing SQL that returns index rather than number-->
  </ConfigItem>
  <!--______________-->
  <!--CLASSIFICATION-->
  <ConfigItem Ignore="true" McField="Type." Type="CUSTOM String/List" PvdExpression="Select CUSTOM_VALUES_SHORTSTR.&quot;value&quot;&#xD;&#xA;From MOVIES &#xD;&#xA;Inner Join CUSTOM_VALUES_SHORTSTR On MOVIES.&quot;mid&quot; = CUSTOM_VALUES_SHORTSTR.&quot;mid&quot;&#xD;&#xA;Inner Join CUSTOM_FIELDS_MOVIES On CUSTOM_VALUES_SHORTSTR.&quot;cmfid&quot; =&#xD;&#xA; CUSTOM_FIELDS_MOVIES.&quot;cmfid&quot;&#xD;&#xA;Where MOVIES.&quot;mid&quot; =@id and CUSTOM_FIELDS_MOVIES.&quot;dispname&quot; = 'Work type'">
    <!--Illustration of custom SHORT TEXT, SELECT or MULTISELECT LIST field (field is referenced by "dispname")-->
  </ConfigItem>
  <ConfigItem McField="Genre." Type="List" Sep="; " PvdExpression="Select GENRES.&quot;value&quot; from GENRES &#xD;&#xA;Inner join MOVIEGENRES on (GENRES.&quot;gid&quot; = &quot;MOVIEGENRES&quot;.&quot;gid&quot;) &#xD;&#xA;Where MOVIEGENRES.&quot;mid&quot; = @id">
    <!--Replacing standard string field with list-->
  </ConfigItem>
  <ConfigItem Ignore="true" McField="Moods." Type="CUSTOM String/List" Sep="; " PvdExpression="Select CUSTOM_VALUES_SHORTSTR.&quot;value&quot;&#xD;&#xA;From MOVIES inner join CUSTOM_VALUES_SHORTSTR On MOVIES.&quot;mid&quot; = CUSTOM_VALUES_SHORTSTR.&quot;mid&quot; inner join CUSTOM_FIELDS_MOVIES on CUSTOM_VALUES_SHORTSTR.&quot;cmfid&quot; = CUSTOM_FIELDS_MOVIES.&quot;cmfid&quot;&#xD;&#xA;Where MOVIES.&quot;mid&quot; =@id and CUSTOM_FIELDS_MOVIES.&quot;dispname&quot; = 'Moods'">
    <!--Illustration of custom MULTISELECT LIST field (field is referenced by "dispname")-->
  </ConfigItem>
  <ConfigItem McField="Keywords" Type="List" Sep="; " PvdExpression="Select TAGS.&quot;value&quot; from TAGS &#xD;&#xA;Inner join MOVIETAGS on (TAGS.&quot;tid&quot; = &quot;MOVIETAGS&quot;.&quot;tid&quot;) &#xD;&#xA;Where MOVIETAGS.&quot;mid&quot; = @id" />
  <ConfigItem McField="Country" Type="List" Sep="; " PvdExpression="Select COUNTRIES.&quot;value&quot; from COUNTRIES &#xD;&#xA;Inner join MOVIECOUNTRIES on (COUNTRIES.&quot;cid&quot; = &quot;MOVIECOUNTRIES&quot;.&quot;cid&quot;)&#xD;&#xA;Where MOVIECOUNTRIES.&quot;mid&quot; = @id" />
  <ConfigItem McField="Studios" Type="List" Sep=";" PvdExpression="Select STUDIOS.&quot;value&quot;&#xD;&#xA;From STUDIOS inner join MOVIESTUDIOS on MOVIESTUDIOS.&quot;sid&quot; = STUDIOS.&quot;sid&quot; inner join&#xD;&#xA;  MOVIES on MOVIES.&quot;mid&quot; = MOVIESTUDIOS.&quot;mid&quot;&#xD;&#xA;Where MOVIES.&quot;mid&quot; =@id" />
  <!--_______-->
  <!--CREDITS-->
  <ConfigItem McField="Producer" Type="List" Sep="; " PvdExpression="Select PEOPLE.&quot;name&quot; from PEOPLE &#xD;&#xA;Inner join CAREER on (PEOPLE.&quot;pid&quot; = &quot;CAREER&quot;.&quot;pid&quot;)&#xD;&#xA;Where CAREER.&quot;mid&quot; = @id AND CAREER.&quot;ctype&quot; = 4" />
  <ConfigItem McField="Screenwriter" Type="List" Sep="; " PvdExpression="Select PEOPLE.&quot;name&quot; from PEOPLE &#xD;&#xA;Inner join CAREER on (PEOPLE.&quot;pid&quot; = &quot;CAREER&quot;.&quot;pid&quot;)&#xD;&#xA;Where CAREER.&quot;mid&quot; = @id AND CAREER.&quot;ctype&quot; = 2" />
  <ConfigItem McField="Director" Type="List" Sep="; " PvdExpression="Select PEOPLE.&quot;name&quot; from PEOPLE &#xD;&#xA;Inner join CAREER on (PEOPLE.&quot;pid&quot; = &quot;CAREER&quot;.&quot;pid&quot;)&#xD;&#xA;Where CAREER.&quot;mid&quot; = @id AND CAREER.&quot;ctype&quot; = 1" />
  <ConfigItem McField="Actors" Type="List" Sep=";" PvdExpression="Select PEOPLE.&quot;name&quot; &#xD;&#xA;From PEOPLE inner join CAREER on (PEOPLE.&quot;pid&quot; = &quot;CAREER&quot;.&quot;pid&quot;)&#xD;&#xA;Where CAREER.&quot;mid&quot; = @id and CAREER.&quot;ctype&quot; = 0"></ConfigItem>
  <ConfigItem Ignore="true" McField="Actors" Type="List" Sep=";" PvdExpression="Select PEOPLE.&quot;name&quot; || '\' || CAREER.&quot;role&quot;&#xD;&#xA;From PEOPLE Inner join CAREER on (CAREER.&quot;pid&quot; = PEOPLE.&quot;pid&quot;)&#xD;&#xA;Where CAREER.&quot;ctype&quot; = 0 And CAREER.&quot;mid&quot; = @id">
    <!--Use nested list-->
  </ConfigItem>
  <ConfigItem McField="Composer." Type="List" Sep="; " PvdExpression="Select PEOPLE.&quot;name&quot; from PEOPLE &#xD;&#xA;Inner join CAREER on (PEOPLE.&quot;pid&quot; = &quot;CAREER&quot;.&quot;pid&quot;)&#xD;&#xA;Where CAREER.&quot;mid&quot; = @id AND CAREER.&quot;ctype&quot; = 3">
    <!--Use list field (as for other people) instead of standard string field-->
  </ConfigItem>
  <!--___________-->
  <!--DESCRIPTION-->
  <ConfigItem McField="Tag Line" PvdExpression="Select MOVIES.&quot;tagline&quot; &#xD;&#xA;From MOVIES &#xD;&#xA;Where MOVIES.&quot;mid&quot;=@id" />
  <ConfigItem McField="Description" PvdExpression="Select MOVIES.&quot;description&quot; &#xD;&#xA;From MOVIES &#xD;&#xA;Where MOVIES.&quot;mid&quot;=@id" />
  <ConfigItem Ignore="true" McField="Review." Type="CUSTOM Memo" PvdExpression="Select CUSTOM_VALUES_LONGSTR.&quot;value&quot;&#xD;&#xA;From MOVIES&#xD;&#xA;Inner join CUSTOM_VALUES_LONGSTR on MOVIES.&quot;mid&quot; = CUSTOM_VALUES_LONGSTR.&quot;mid&quot; &#xD;&#xA;Inner join CUSTOM_FIELDS_MOVIES on CUSTOM_FIELDS_MOVIES.&quot;cmfid&quot; = CUSTOM_VALUES_LONGSTR.&quot;cmfid&quot;&#xD;&#xA;Where MOVIES.&quot;mid&quot;=@id and CUSTOM_FIELDS_MOVIES.&quot;dispname&quot; = 'Review'">
    <!--Illustration of custom MEMO field-->
  </ConfigItem>
  <!--______-->
  <!--RATING-->
  <ConfigItem McField="MPAA Rating" PvdExpression="Select MOVIES.&quot;mpaa&quot; &#xD;&#xA;From MOVIES &#xD;&#xA;Where MOVIES.&quot;mid&quot;=@id" />
  <ConfigItem McField="Rating.IMDb" Type="Decimal" PvdExpression="Select MOVIES.&quot;imdbrating&quot; &#xD;&#xA;From MOVIES &#xD;&#xA;Where MOVIES.&quot;mid&quot;=@id">
    <!--Use in expression (column or field) to display as rating stars-->
  </ConfigItem>
  <ConfigItem Ignore="true" McField="Rating.AllMovie" Type="CUSTOM Real number (Decimal)" PvdExpression="Select CUSTOM_VALUES_REAL.&quot;value&quot;&#xD;&#xA;From MOVIES &#xD;&#xA;Inner Join CUSTOM_VALUES_REAL On MOVIES.&quot;mid&quot; = CUSTOM_VALUES_REAL.&quot;mid&quot; &#xD;&#xA;Inner Join CUSTOM_FIELDS_MOVIES On CUSTOM_VALUES_REAL.&quot;cmfid&quot; = CUSTOM_FIELDS_MOVIES.&quot;cmfid&quot;&#xD;&#xA;Where MOVIES.&quot;mid&quot; =@id and CUSTOM_FIELDS_MOVIES.&quot;dispname&quot; = 'AllMovie rating'">
    <!--Illustration of custom rating (and any other custom REAL NUMBER) field-->
  </ConfigItem>
  <ConfigItem Ignore="true" McField="Rating.Other" Type="Decimal" PvdExpression="Select MOVIES.&quot;orating&quot; &#xD;&#xA;From MOVIES &#xD;&#xA;Where MOVIES.&quot;mid&quot;=@id" />
  <ConfigItem Ignore="true" McField="Rating.User" Type="Decimal" PvdExpression="Select MOVIES.&quot;rating&quot; &#xD;&#xA;From MOVIES &#xD;&#xA;Where MOVIES.&quot;mid&quot;=@id" />
  <!--__________-->
  <!--STATISTICS-->
  <ConfigItem Ignore="true" McField="Budget." Type="Integer" PvdExpression="Select MOVIES.&quot;budget&quot; &#xD;&#xA;From MOVIES &#xD;&#xA;Where MOVIES.&quot;mid&quot;=@id" />
  <ConfigItem Ignore="true" McField="Box office." Type="Integer" PvdExpression="Select MOVIES.&quot;money&quot; &#xD;&#xA;From MOVIES &#xD;&#xA;Where MOVIES.&quot;mid&quot;=@id" />
  <ConfigItem Ignore="true" McField="Viewed." Type="Date string" PvdExpression="Select MOVIES.&quot;viewdate&quot; &#xD;&#xA;From MOVIES &#xD;&#xA;Where MOVIES.&quot;mid&quot;=@id">
    <!--Use expression to convert format to YYYY-MM-DD for correct sorting-->
  </ConfigItem>
  <!--_________________-->
  <!--MEDIA INFORMATION-->
  <ConfigItem Ignore="true" McField="Duration." Type="CUSTOM Number (Integer)" PvdExpression="Select CUSTOM_VALUES_INT.&quot;value&quot;&#xD;&#xA;From MOVIES &#xD;&#xA;Inner Join CUSTOM_VALUES_INT On MOVIES.&quot;mid&quot; = CUSTOM_VALUES_INT.&quot;mid&quot; &#xD;&#xA;Inner Join CUSTOM_FIELDS_MOVIES On CUSTOM_VALUES_INT.&quot;cmfid&quot; = CUSTOM_FIELDS_MOVIES.&quot;cmfid&quot;&#xD;&#xA;Where MOVIES.&quot;mid&quot; =@id and CUSTOM_FIELDS_MOVIES.&quot;dispname&quot; = 'OrigDuration'">
    <!--Illustration of custom NUMBER field;
Standard field not used because it's formated HH:MM:SS and can't be edited-->
  </ConfigItem>
  <ConfigItem Ignore="true" McField="MediaInfo.Video" Type="CUSTOM String" PvdExpression="Select VCODECS.&quot;value&quot; || ', ' || MOVIES.&quot;resolution&quot; || ', ' ||&#xD;&#xA;  MOVIES.&quot;videobitrate&quot; || ' Kbps'&#xD;&#xA;From MOVIES inner join VCODECS on VCODECS.&quot;vcid&quot; = MOVIES.&quot;videocodec&quot;&#xD;&#xA;Where MOVIES.&quot;mid&quot;=@id">
    <!--Illustration of custom string CONCATENATION-->
  </ConfigItem>
  <ConfigItem Ignore="true" McField="MediaInfo.Audio" Type="CUSTOM String" PvdExpression="Select ACODECS.&quot;value&quot; || ', ' || AUDIO.&quot;bitrate&quot; || ' Kbps'&#xD;&#xA;From MOVIES inner join AUDIO On MOVIES.&quot;mid&quot; = AUDIO.&quot;mid&quot; inner join ACODECS on ACODECS.&quot;acid&quot; = AUDIO.&quot;acid&quot;&#xD;&#xA;Where MOVIES.&quot;mid&quot;=@id"></ConfigItem>
</Organizer>

I included most of the standard fields included in PVD, illustrations of each of the four different types of custom fields, and a few tricks. Note that I always use a "." in my custom field names—to make them distinct from MC standard fields. So "Name" is the MC standard field Name, while "Year." is a custom field.

Quote
Maybe you and Darichman could cooperate on this?

I hope he, and anyone else who is interested will test this, offer comments and suggest additions. While this covers most of the different fields available, it would be nice to include more options for handling some of the items. For example, I've included the obvious query for Actors, but also darichman's brilliant alternative of creating a nested list with Actor\Role.
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 22, 2009, 10:36:14 am
Yeah, really sorry I haven't been around much lately... I'm getting a bit overwhelmed with things away from the computer (yes, I know, shock horror!!). Have my final set of exams coming up in the not too distant future... eek! :-\

Looking at the change logs for the last few versions it really looks like things are coming together! I'll put in some time in the next week or so - I agree a nice template to begin with, for users to customise the plugin to suit their own fields etc is a great idea.

Alrite, its half past one over here. Bed time for me!

Cheers
Chris
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 22, 2009, 12:13:06 pm
*********************
Known Issues 1.0.12:
*********************
o If you select too many files (in the hundreds) in "Send To...", the PvdImportContextMenu.exe applications may get confused. Select fewer (50, maybe)!

1.0.12:
----------
o Changed: Some more comments in the user manual
o Changed: Several new fields/rows and corresponding sql+description, thanks to rick.ca
o Feature: New Description column in Field Configuration: Contains description of rows
o Feature: New checkbox which allows the user to hide Description column
o Changed: Some cosmetics in field configuration + corresponding tooltip changes
o Changed: "Sucess" -> "Success" :)
o Changed: Internal stuff: Databinding of all controls now works according to how they're supposed to
o Changed: Internal stuff: Split view code from data handling code. Better structure of data handling code
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 22, 2009, 12:48:35 pm
I included most of the standard fields included in PVD, illustrations of each of the four different types of custom fields, and a few tricks. Note that I always use a "." in my custom field names—to make them distinct from MC standard fields. So "Name" is the MC standard field Name, while "Year." is a custom field.
That dot is actually a pretty good idea. It avoids fields being grabbed by MC - also it's easier to see which fields "belong to you".

I changed the default MC user field names so that the "." is in the beginning of the field name, sorry about that, Rick.ca ! The reason is that I'd like some "more system" in the naming + it's less likely that JRiver adds a field with a "." in the beginning sometime in the future.

I know i said "no more features" but the Description field seemed neccessary...
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 22, 2009, 02:47:12 pm
It seems the posted version is still 1.0.11. :(

Quote
I changed the default MC user field names so that the "." is in the beginning of the field name, sorry about that, Rick.ca!

That's okay—users are going to have their own preference as to how to handle this. The point is, one needs to have a way of distinguishing custom field display names. Even though the display name can be anything—including the same as that used for the standard field—only the display name appears in lists. So, "Genre" becomes "Genre.", and the "." is reasonably unobtrusive when it appears in a column header (or wherever). ".Genre" works just as well, except for one thing: ".Genre" and "Genre" will not appear together in lists. This matters, as it's often necessary to appreciate both exist in order to choose the one that is appropriate in the circumstances.

Your "more system" consideration, however, is important as well. Maybe it's better to do it this way just to be really clear in distinguishing custom from standard fields in the default. Users will have to create any custom fields they want to use anyway—so they can change the display name according to their own preferences as they do so. BTW, while it's not directly relevant to the plugin, it's probably good practise to name all custom fields "pvd.Fieldname" (with a display name of ".Fieldname" or "Fieldname.")—to distinguish them from custom fields added for other reasons.

Quote
o Changed: Several new fields/rows and corresponding sql+description, thanks to Chris.ca

Who is Chris.ca? ;)
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 22, 2009, 03:03:15 pm
It seems the posted version is still 1.0.11. :(
Sometimes, you're really fast :) . You must have downloaded it between me posting on the forum and uploading to the site...

It should be the correct version now, though.

Who is Chris.ca? ;)

Sorry :)
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 22, 2009, 05:08:42 pm
Quote
Sometimes, you're really fast Smiley . You must have downloaded it between me posting on the forum and uploading to the site...

Nope. It seems the problem was Firefox (or Windows?) was getting the previously downloaded file from cache. Rather than using the link, I had to right-click, select "save link as..." and give it a different name. It would be helpful if you could include the version number in the installer filename. That also makes it easier for those of us who like to keep a few versions of releases.

Quote
I know i said "no more features" but the Description field seemed neccessary...

Perfect! As I was working on it, I wondered if you could do something like this, but thought I'd better respect your "no more features" decision. I might have known that would be overridden by "do the right thing." ;D 

I think this will work very well. It will be an easy way do document the purpose of, and provide instruction for the use of the queries provided. It will also provide the user with the means to document their own modifications.

For custom fields, it would be helpful to indicate the recommended Data Type of the field the user is to create. In the interest of brevity, it could be left out if it's the default String. Otherwise, for example:

     McField     Description
     .Genre     McField: User (List)
     .Rating IMDb     McField: User (Decimal)
     .Budget     McField: User (Integer)

I think it's important to avoid the MC default Year field. Unless I've managed to get myself (and/or my MC configuration) confused, this is the display name for the Date (year) field—a calculated field based on the file date modified. So whatever is put in the Year McField will be overwritten by the year according to the date modified file stamp.
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 22, 2009, 06:26:05 pm
I think it's important to avoid the MC default Year field. Unless I've managed to get myself (and/or my MC configuration) confused, this is the display name for the Date (year) field—a calculated field based on the file date modified. So whatever is put in the Year McField will be overwritten by the year according to the date modified file stamp.
Yeah, I realized this after I'd uploaded the file.

And I'll add those recommended field types.
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 23, 2009, 08:38:56 am
Rick's excel doc has inspired me to go ahead and build a config file with most of the fields PVD can pull from IMDb and AMG. My set-up is similar to Rick's. My field name setup isn't very standardised, however... It's probably a bit over-inclusive, as it includes just about every custom field you can pull from AMG and IMDb, using the import plugins Rick and I developed a while ago.

I've also put together a rather colourful (and sortable) excel document which shows which fields are matched to what. My field names are a little different than Rick's, both in MC and PVD, so there are some slight changes to the corresponding SQL queries. It should be easy enough for the user to change the relevant bits to suit their field names, however. I'll also post the PVD settings I used to get the info (import plugin and custom fields, so anyone can replicate it if they need to)

Just a heads up, the plugin doesn't work with MC14, although there's probably not a lot we can do about that at the moment... when you install it into the MC14 plugin directory it doesn't appear in the plugin manager (and you can't import the *.dll's manually either). Have you locked it to a particular version? Other interface plugins I had installed for 13 were picked up automatically by 14. Looks like I'll be version hopping for a bit! - but I can definitely justify it to use this great plugin.

As far as using "Year" goes... "Year" in MC refers to the [Date (Year)] field which is, as you say, used when showing "Date". It is not automatically modified from the file timestamp if you manually put another value in there. I've used the standard Year field for all my movies for a while - I guess it's just a matter of what you prefer.

Thanks again guys for the hard work - it's all taking shape now :)
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 23, 2009, 04:14:16 pm
Quote
I've also put together a rather colourful (and sortable) excel document which shows which fields are matched to what.

My comments and questions on your worksheet:


Query for Original Language:

Code: [Select]
Select LANGUAGES."value"
From MOVIES inner join LANGUAGES On MOVIES."origlang" = LANGUAGES."lid"
Where MOVIES."mid" =@id

It's not clear to me what you think should be included in the default field configuration. I believe we need to be careful not to overwhelm the new user (to PVD, as well as the plugin) by including too many queries they are unlikely to use. This is the process I imagine a typical new user is likely to go through:

1. Install and configure PVD in the simplest possible way—using standard fields only, populating them with IMDb only.

2. Install and configure PvdImport. Find that all standard PVD fields are already configured, although some require the creation of custom MC fields to accommodate the data. Create these custom fields according to tips (e.g., data type) included in the row descriptions. Create a view scheme to display this information.

3. Learn that meta data from other sources (e.g., AllMovie) may be added to, or combined with (i.e., for some elements, use one source rather than the other) the IMDb information. Create custom fields to accommodate additional information. Configure overwrite settings to effect desired combination of sources. This step takes some time for one to rationalize their personal preferences as to what information to collect, and this has nothing to do with the PvdImport configuration (i.e., it just needs to accommodate the final result).

4. Add custom fields to PvdImport configuration to accommodate the PVD custom fields added in 3. Create the necessary query by copying and modifying the sample provided for the same data type. Add the corresponding custom field to MC (or map it to an unused standard field, if that works) and incorporate it into view schemes.

5. At this point, it should be appreciated the configuration is not nearly as intimidating as it probably seemed initially. Modifications are easy to make, by direct editing of queries, or using the Ignore feature to switch between alternate queries.

Thus my suggestion the default configuration file should avoid creating any confusion for those at step 2, while providing helpful options and illustrations for those at step 4. Also, I think it's important to make it clear what is a PVD configuration issue (including all matters of personal preference as to what meta data is collected and how) versus a plugin or MC configuration issue. I hope people will go to PVD for the former, and here for the latter.

A few suggestions come out of this:

1. Now that the queries are ordered in a logical fashion, it would be nice to be able to change the order from within the plugin, and/or be able to insert a new query rather than add it to the end of the list. (For those of use who care about such things, however, this is very easy to do using a XML editor.)

2. Maybe the queries provided as an illustration of how to handle a custom data type should be grouped together at the end of the list—so it's clear this is what they are for.

Quote
Just a heads up, the plugin doesn't work with MC14...

Did I read something about a new import facility? If so, will this have any impact on the plugin, or user's perception of it's usefulness?
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 23, 2009, 11:14:18 pm
Thanks for the updated queries :)

Are you sure Year "is not automatically modified from the file timestamp if you manually put another value in there"? I haven't actually tested it, but my concern is MC may "detect external changes" (or whatever) and update the field based on a new Date Modified.

I'm not a developer, but this is my understanding (disclaimer: based more on experience than actual knowledge of what's going on under the hood!)

[Date] is filled out automatically from the file stamp on first import of any file, unless the file has an embedded tag which contains date information (eg EXIF for images, ID3 for mp3 etc). Unfortunately with video files, embedded tags aren't well supported by any media players that I know of, so any video you import with MC will use the timestamp as the [Date] field by default. [Date] is made up of [Date (Year)] etc but these can be edited independently. Typically, I clear the [Date] field altogether (to remove date and time values) and then just type in a year. The good thing is, once you clear the date field or replace it with another value (eg the release year for a movie) it's there to stay, as it's stored in the database. The time-stamp of a video file won't really change in most circusmtances - Auto-import etc will not change the [Date] fields once you've put a value in there... I've done this for years now with no issues.

The advantage of using the default date fields over a custom one is that Sort by Date etc works natively with default views/sort options.

Quote
It may seem unlikely a media file would change, but maybe the effect would be the same even if the file was just moved.

If a file is moved or renamed, MC loses it's connection to the file. If an auto-import is done or the new file is imported manually, MC will import the renamed/moved file as a new file, as it has no way of knowing that it is related to an existing file in the database. This means all metadata will be blank, apart from the default [Date] (based on timestamp) described above -- the file is essentially a new file. The exception is for files which are taggable (eg MC will detect if tagged mp3s or photos have been moved, although these already contain readable metadata so the point is moot I guess).

This affects all fields, so whether you're using the default date fields or a custom one isn't important.

Quote
Production Credits consists of HTML links—have you found a way to use this in MC?

My original idea was to use the information in this field, as is, in a track info template - which is really just a generated html page anyway. Then I remembered that you can't use the track info plugin with video (as the file needs to be playing to see it, and with video files the display will naturally be the... err video. So that was a dead end. I would like to have this information, even just as plain text without the link functionality in an extended note field... but haven't been able to come up with a way to filter out the link stuff. So it's just in there to remind me it still exists. It's ignored by default I think.

Quote
How do you use your "Links" fields in MC—particularly URL, which typically contains more than one URL?

My "URL" field from PVD only contains the IMDb link (I use a custom one for AMG) - so I have two corresponding custom fields in MC, both string. People have asked for a new field type, "Link" to be available, which would make custom fields like these launchable (either with the existing internal browser interface, or by launching an external browser window). The other thing I was trying to do was use the inbuilt Links feature (in the top right of any view) to launch the url in these fields - I didn't have any luck with this, as I was only able to get it to search, not open a direct URL. Grr... so, I guess these are in there just for storage at the moment, until I figure out how to get either of the above ideas to work!

Quote
How do you use your "Genre / Genre 1 / AMG Genres" fiield?

I still haven't decided which Genre system to use for movies (either IMDb's or AMG's). This accounts for my separate [AMG Genres] and [IMDb Genres] fields. AMG's genres tend to me more strictly defined, so I am using those by default at the moment.

I am only using the default [Genre] field temporarily, as it is bizarrely a string field (not a list) but at least displays in the current File Info panel in theatre view. Once the file info panel is customisable, this will go away for good ;)

I have detailed my Type and Genre system elsewhere... briefly I have two type fields [Type 1] & [Type 2] and three genre fields [Genre 1], [Genre 2] and [Genre 3] on which my entire media library is based (not just videos). [Genre 1] is my broadest level, and is currently the only level I'm using for movies (although I was thinking of mapping AMG's type field to [Genre 2], as it's really a more specific classification of genre)

[Type 1] and [Type 2] are used as filters for creating all of my view schemes etc. Eg. [Type 1] might be Film , with [Type 2] as Feature Film, Trailer, Soundtrack etc. Both levels are lists so a soundtrack will have a [Type 1] of Music;Film and a [Type 2] of Film Soundtrack. This allows me, for example, to see all media related to a particular movie (music, pictures, sheet music, trailers, special features, screenplay etc as well as the movie itself) when I click on a movie name. Alternatively, I can limit a view to "Feature Film" which would only show me the actual movie files themselves.

Quote
It's not clear to me what you think should be included in the default field configuration. I believe we need to be careful not to overwhelm the new user (to PVD, as well as the plugin) by including too many queries they are unlikely to use.

I do not think mine should be the default configuration. I think we should have a default which contains only the default fields which PVD can grab (ie without any customisation). As you say, exemplary custom field queries could be included at the bottom of the plugin page, ignored by default. This should be the default config file used by PVDImport on a new install.

I also think it's worthwhile including a second config file in the installation folder with an exhaustive list of fields which users can select from. After all, it's easier to delete the ones you don't want than create a million new ones. If this is done, we do need to recognise that no system is the best system and, as you say, people have different preferences for both sources of information and field mappings.

I've made my import plugin (AMG/IMDb) and PVD settings available, as has Rick, so advanced users can replicate our setups if they want to, but I suspect most users will prefer to do things in a way which best suits their individualised needs. Provide the option - a quick, easy default setup which requires little to no customisation, and an advanced option... which would probably require some sort of hand-holding through how the fields need to be set up/imported...

I'd like to post a "MC Users Guide to getting information from PVD  using PVDImport - This could include all the steps you mention. I think the key issue is field mapping and this occurs at two levels (Internet Source > PVD ... and.... PVD > MC). There is plenty of opportunity for confusion, especially if the user is overwhelmed with massive spreadsheets/config fields with every conceivable field. I have documented my setup with screenshots etc as well, so it shouldn't be too hard to put something together... time is the issue though :P

Some feature suggestions, although I have no idea whether the plugin implementation in MC makes these possible or not (or how much work it would take on raldo's end ;))

Quote
Did I read something about a new import facility? If so, will this have any impact on the plugin, or user's perception of it's usefulness?

No, the change is related to importing files, not metadata (basically detecting folders suitable for import based on a few settings).
Pvd will still retain its usefulness, and will for some time, I'm sure...
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 23, 2009, 11:39:46 pm
EDIT: I just noticed a problem. The custom short string values (Moods, Tones, Themes etc) in the config file I uploaded output with a comma "," as the separator, not a semicolon. Something's afoot...!

I think this is because I stored most of my custom fields as "Short Text" fields in PVD. This worked fine with our old MPL export templates which could replace "," with ";". But naturally with PvdImport these are outputting as a string, not as separate values so the ; delimiter is not getting applied. *Bangs head on desk for not realising this sooner*

EDIT Again: I changed the relevant custom fields to Select List, but I'm still seeing this.
Have I stumbled across a problem here, or am I just missing something obvious?

Rick, does your query for Moods return commas or semicolons on your setup?

Code: [Select]
Select CUSTOM_VALUES_SHORTSTR."value"
From MOVIES inner join CUSTOM_VALUES_SHORTSTR On MOVIES."mid" = CUSTOM_VALUES_SHORTSTR."mid" inner join CUSTOM_FIELDS_MOVIES on CUSTOM_VALUES_SHORTSTR."cmfid" = CUSTOM_FIELDS_MOVIES."cmfid"
Where MOVIES."mid" =@id and CUSTOM_FIELDS_MOVIES."dispname" = 'Moods'
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 24, 2009, 05:47:55 am
Quote
Auto-import etc will not change the [Date] fields once you've put a value in there...

I think we both got it wrong.  :o

You convinced me I should be able to use the standard Year field. But when I tried, I found it could not be overwritten. I think it's just a pseudo field that reflects the year of Date. It could be updated by importing a Date, but we don't have the proper date value. It seems importing is different than manually entering something—that will, as you say, will be stored in the database and not changed by anything else.

It occurs to me that if we could convert Release date to an MC date value, it could be saved as Date and that would provide Year as well. The problem with this is the year of release is not always the year of the movie. Sometimes a movie is completed in the year before release, and users may use a later release date (e.g., that for their own location).

So I think we need to accept "movie years" are different from "MC years," and stick with a custom Year field for video.

Quote
If a file is moved or renamed, MC loses it's connection to the file.

True, but the plugin has allow MC to finally handle my strange work flow. I tend to download, watch and delete. The "delete" is done with a script that creates a null filename.log file, moves it to a "viewed" folder, and then deletes filename.avi. Filename is "Title (Year)" and the date modified is soon after viewing, so my "viewed" folder is sort of a log of my viewing. The file, of course, is "gone" to MC. But the PVD scanner records the change in filename and path. MC picks up the log as a new file, and then the plugin re-imports the meta data. This way, I get to keep all the information about the movies I've seen, whether I keep them or not. It's convoluted, but more or less automatic.

Quote
I have detailed my Type and Genre system elsewhere...

I thought it sounded familiar. Thanks for the recap. ;)

Quote
I also think it's worthwhile including a second config file in the installation folder with an exhaustive list of fields which users can select from.

If we provide for standard fields and an example of every possible kind of query, that should be enough. Providing more custom fields just glosses over the fact the user can add whatever they want, and there are only four different query forms needed for them. Or to look at it another way: First, users have to add a custom field to PVD and make sure it's properly populated (e.g., by a script). Then a corresponding field has to be added to MC and format suitability considered (e.g., perhaps an expression is necessary to display it properly). Having gone this far, it's probably easier to copy one of four queries and edit as required, rather than searching "an exhaustive list" for one that best fits—but likely still needs editing.

Anything beyond the most straightforward configuration (which may still be more that what many users need) reaches into areas of pure personal preference. What would help (in a future version) is a field mapping facility that would detect all fields in the PVD database, provide a query, and allow a destination MC field to be matched or created for each one. But even this would have to be kept simple and flexible, recognizing, for example, some users will want to concatenate and map fields in unconventional ways.

Let's not forget there's probably still some room for including some optional queries in the default configuration—not for additional fields, but for different ways of handling the information (e.g., the nested Actor\Roles field). I've figured out a way of handling series that works for me, but I realize people are going to want to do that different ways. Maybe we can provide more tools to help make that easier.

Quote
EDIT Again: I changed the relevant custom fields to Select List, but I'm still seeing this.
Have I stumbled across a problem here, or am I just missing something obvious?

A Select List is a field where you select one item from a list—the result is a string. What you want is a Multiselect List—the PVD equivalent of MC's semicolon-delimited list. For example, I save Moods in a Multiselect List and the query works fine—the items are semicolon delimited.
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 24, 2009, 06:58:44 am
I think we both got it wrong.  :o

You convinced me I should be able to use the standard Year field. But when I tried, I found it could not be overwritten. I think it's just a pseudo field that reflects the year of Date. It could be updated by importing a Date, but we don't have the proper date value. It seems importing is different than manually entering something—that will, as you say, will be stored in the database and not changed by anything else.

Set McField to: Date (Year)   (NOT Year)

And use the usual query:

Code: [Select]
Select MOVIES."year"
From MOVIES
Where MOVIES."mid"=@id

That seems to work okay!

Quote
Let's not forget there's probably still some room for including some optional queries in the default configuration—not for additional fields, but for different ways of handling the information (e.g., the nested Actor\Roles field). I've figured out a way of handling series that works for me, but I realize people are going to want to do that different ways. Maybe we can provide more tools to help make that easier.

I was thinking of getting a bit creative and trying to do something to combine all of the "People" fields (Director, Producer, Actor etc) in some useful way. This could be useful for people who have had multiple roles, both acting and production-wise, in films.

If we use queries like:

Code: [Select]
Select PEOPLE."name" || '\' || 'Director'
From PEOPLE Inner join CAREER on (CAREER."pid" = PEOPLE."pid")
Where CAREER."ctype" = 1 And CAREER."mid" = @id

and

Code: [Select]
Select PEOPLE."name" || '\' || 'Producer'
From PEOPLE Inner join CAREER on (CAREER."pid" = PEOPLE."pid")
Where CAREER."ctype" = 4 And CAREER."mid" = @id

is there any way to link the output of the query so we get: [SQL Result A];[SQL Result B] ?

If we could do this we could emulate the works list you get in PVD when you click on a person (ie, the acting roles they've been in as well as any directorial/production roles)
Might be useful in creating "People" focused views in MC. Just a thought.

Quote
A Select List is a field where you select one item from a list—the result is a string. What you want is a Multiselect List—the PVD equivalent of MC's semicolon-delimited list. For example, I save Moods in a Multiselect List and the query works fine—the items are semicolon delimited.

Wow, you'd have thought I'd have picked that up by now. I guess my old way of exporting was masking this for me. Thanks for clearing it up... now to run all the imports in PVD again :)
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 24, 2009, 12:22:49 pm
Quote
Set McField to: Date (Year)   (NOT Year)

OIC. It's the field name Date (year) with the display name Year. Duh! :-[

Quote
is there any way to link the output of the query so we get: [SQL Result A];[SQL Result B] ?

You could do an MC ListCombine of the separate results, but I haven't yet come across a way to combine separate queries into one. That would be less "messy," and would have a more general application—for combining anything that doesn't really need to be in separate fields.
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 24, 2009, 09:56:15 pm
You could do an MC ListCombine of the separate results, but I haven't yet come across a way to combine separate queries into one. That would be less "messy," and would have a more general application—for combining anything that doesn't really need to be in separate fields.

We seem to be on the same page with a lot of things ;) I did consider this - in my experience with the listcombine feature, it can really slow down large views. If we put nesting in the mix as well... We can only try it out I guess.

You seem pretty nifty with concatenation - I shamelessly copied your audio/video information queries ;)
This just adds another step (the "\Role" part after each entry). I wonder if there's another way to get an output of: Directorname\Director;Producername\Producer;Actorname\Role etc etc
Title: Re: Personal Video Database Import plugin - Test log
Post by: Daydream on May 25, 2009, 12:51:54 am
Guys, in case you'll worry later, where did the camel's back break, I believe it's my duty to point that that time is _now_, when you started using random inner joins between fields to make the plugin output data in a custom way. :)
I believe this is called the JRiver symptom, and it manifests by a logarithmic explosion of database usage styles, the moment you got your hands on the tools. Doesn't matter what's out there - me, myself and my custom taste rule the world! :)

Please divide time for writing a help file!  ;D
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 25, 2009, 01:38:49 am
We seem to be on the same page with a lot of things ;) I did consider this - in my experience with the listcombine feature, it can really slow down large views. If we put nesting in the mix as well... We can only try it out I guess.

Yes, I've already discovered I have a serious problem just selecting "too many" records at the same time from my default "everything" view. I suppose MC is trying to read the whole works into memory. I get around this by using a simpler view for operations requiring the selection of a large number of records.

Quote
I wonder if there's another way to get an output of: Directorname\Director;Producername\Producer;Actorname\Role etc etc

There must be a way using sub-queries, derived tables or unions (SQL stuff I don't yet understand), but I haven't been able to figure it out. The best I can come up with is...

Code: [Select]
Select PEOPLE."name" || '\' || CAREER."ctype" || '\' || CAREER."role"
From PEOPLE inner join CAREER on (CAREER."pid" = PEOPLE."pid") inner join MOVIES on CAREER."mid" = MOVIES."mid"
Where CAREER."mid" =@id

...the result of which is rather tacky. :-\
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 25, 2009, 01:45:23 am
What's the conclusion on the Year field?! (There's a lot of text above and I am afraid I don't have time to get too involved in your SQL discussions   :) )

Darichman, did you test those extra amg fields in the attachment of your previous post?

If yes, do you guys think a "bloated" config file is ok for initial release?
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 25, 2009, 01:55:23 am
Another issue:

Do you think the MC Name field should be omitted from the config file?

When you initially import a video, this field is populated by mc based on the filename. I usually clean up this field and I've seen that an import from PVD can sometimes have weird effects. for example, if PVD doesn't reckognize the video, you may get any kind of result back. Then it's hard to figure out which file you're dealing with after an import from PVD. Also, PVD swaps the location of articles ("The", etc) to the end of the name...

Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 25, 2009, 01:59:28 am
Guys, in case you'll worry later, where did the camel's back break, I believe it's my duty to point that that time is _now_, when you started using random inner joins between fields to make the plugin output data in a custom way. :)

I think you're exaggerating in suggesting we're there now, but I agree we should be careful not to put ourselves in the middle of users' customization fantasies. Personally, I believe the collection and management of video meta data should be done with PVD—that's what it's designed for. Once that's done, the plugin does a very good job of moving the data into MC on a straightforward 1:1 field mapping basis. As far as I'm concerned, if users then have a problem handling the data in MC—that's an MC issue.
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 25, 2009, 03:03:09 am
What's the conclusion on the Year field?!

It's the field name Date (year) with the display name Year.

Quote
If yes, do you guys think a "bloated" config file is ok for initial release?

No! It should be similar to the current one, with a few modifications and changes...

I do not think mine should be the default configuration. I think we should have a default which contains only the default fields which PVD can grab (ie without any customisation). As you say, exemplary custom field queries could be included at the bottom of the plugin page, ignored by default. This should be the default config file used by PVDImport on a new install.

Quote
When you initially import a video, this field is populated by mc based on the filename. I usually clean up this field and I've seen that an import from PVD can sometimes have weird effects. for example, if PVD doesn't reckognize the video, you may get any kind of result back. Then it's hard to figure out which file you're dealing with after an import from PVD.

I suppose it could go into a custom Title. field—but not because there is any good reason why Name should be something different than Title. I don't agree with the premise the Title in PVD is likely to be incorrect, while Name in MC is correct. If the title in PVD is not exactly what it should be, it can only be because PVD is not correctly configured or updated—user error. But using a separate field and comparing the two might be a helpful way to detect errors. The method of correcting any error, however, would be to fix it in PVD, and then re-import.

Having said that, I personally don't see any point in this. I have no difficulty ensuring the data is correct in PVD—so I would want it to overwrite anything that's different in MC anyway. I think it would be better for the default configuration to put Title into Name, and perhaps include the option of a custom field. The description for the optional query could explain that while PVD's Title should be an appropriate Name, one might prefer to use a custom field so Name can be used as a test for error conditions. Including it, however, may raise more questions than it's worth.

Quote
Also, PVD swaps the location of articles ("The", etc) to the end of the name.

This is configurable in PVD. It's a matter of personal preference, but I can't imagine why anyone would want it one way in PVD, and another in MC.
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 25, 2009, 05:14:42 am
Darichman, did you test those extra amg fields in the attachment of your previous post?

If yes, do you guys think a "bloated" config file is ok for initial release?


1. Yes, I tested all of them, they work fine - apart from Release & Language (an production credits works, it just contains some html stuff too though). Rick corrected my Release one, but I still can't get original language to work
2. No, keep the default one simple. Rick's one from further up would probably suffice (just fix up the Date reference).

Quote
I think we should have a default which contains only the default fields which PVD can grab (ie without any customisation). As you say, exemplary custom field queries could be included at the bottom of the plugin page, ignored by default. This should be the default config file used by PVDImport on a new install.

I did suggest maybe putting an expanded (IMDb +AMG) one in the plugin folder as well, as an option if the user requires it... I've attached my updated one below.

Do you think the MC Name field should be omitted from the config file?

Personally, yes. Perhaps it could be ignored by default.
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 25, 2009, 05:18:17 am
I suppose it could go into a custom Title. field—but not because there is any good reason why Name should be something different than Title. I don't agree with the premise the Title in PVD is likely to be incorrect, while Name in MC is correct. If the title in PVD is not exactly what it should be, it can only be because PVD is not correctly configured or updated—user error. But using a separate field and comparing the two might be a helpful way to detect errors. The method of correcting any error, however, would be to fix it in PVD, and then re-import.

Well, the promise wasn't that it was *likely* to be incorrect. It's actually worse when it's incorrect one out of, say, 100 movies because then you'll more likely miss the error. We can't expect everyone to know the inns and outs of PVD the first time they use it, can we?

For example, say the name in MC is complete and non ambigous, and the lookup in PVD has failed because the filename couldn't be resolved in PVD. After an import, you'll have both the "wrong" or ambigous filename in MC and the wrong Name field.

This happened for 4 out of 500 files for me. I ended up restoring and older DB..

What about resolving it as follows: Put "Ignore" on Name and a warning in the field Description?
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 25, 2009, 05:22:01 am
I did suggest maybe putting an expanded (IMDb +AMG) one in the plugin folder as well, as an option if the user requires it... I've attached my updated one below.

Ok. That's a good idea.
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 25, 2009, 05:23:54 am
Btw, how does 1.0.12 look, did you guys find any snags (apart from the field config stuff)?
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 25, 2009, 06:52:20 am
Quote
We can't expect everyone to know the inns and outs of PVD the first time they use it, can we?

Exactly—this is the main reason for my concern. I know my PVD data is perfect, and all I need to do is get it into MC. Virtually no other users (I'm aware of) have this perspective. Other users are more likely to be familiar with MC, but know little about PVD, and are primarily interested in the plugin as a means of obtaining video information to add to their MC database. From this perspective, titles/names not agreeing suggests a problem or an unreliability that should be addressed by the plugin. I just think you should do what you can to avoid creating that impression.

Quote
What about resolving it as follows: Put "Ignore" on Name and a warning in the field Description?

Yes, that would be fine—with a description/comment something like, "In most situations, the PVD 'Title' would be an appropriate 'Name' in MC. Since MC automatically fills 'Name' from the filename, however, it is recommended 'Title' be imported to a separate custom field. In rare cases where the media has been misidentified or incorrectly updated in PVD, the resulting difference between 'Title' and 'Name' may be useful in identifying the error."

Quote
I still can't get original language to work

It works for me. I don't know why it would make a difference, but this one's not a list field—are you putting the result in a string field?
Edit: Oops. Now I see—it only works in a some cases. Back to the drawing board. ::)

Quote
Btw, how does 1.0.12 look, did you guys find any snags

It seems solid. I didn't go out of my way so test it in ways not applicable to my circumstances, however. For example, the cover art feature works fine for "InTheSameFolderAsTheFile," but I didn't test other options.
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 25, 2009, 01:58:08 pm
Quote
Edit: Oops. Now I see—it only works in a some cases. Back to the drawing board.

Query for Original Language:
Code: [Select]
Select LANGUAGES."value"
From MOVIES inner join LANGUAGES on MOVIES."origlang" = LANGUAGES."lid"
Where MOVIES."mid" =@id

I had MOVIES."num" rather than MOVIES."mid". Apparently, details matter. :-[
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 25, 2009, 04:11:42 pm
*********************
Known Issues 1.0.13:
*********************
o If you select too many files (in the hundreds) in "Send To...", the PvdImportContextMenu.exe applications may get confused. Select fewer (50, maybe)!

1.0.13:
----------
o Changed: Updated field configuration to contain Darichman's suggestions too.
o Feature: New checkbox "Ignore". When selected all fields marked Ignore are displayed, otherwise, none marked ignore are displayed.

I.e., the default field configuration file is a "Maximum version", but the "Ignore" checkbox makes the display less intimidating when PvdImport is first started..

Sorry, Rick.ca, no version number in the file name. I'm already doing about 10 steps to get every release out there...

Any comments on the field config file? If yes, please post suggestions with the original field names. (Then I can use a diff tool to verify the changes)



Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 25, 2009, 04:21:50 pm
Quote
I wonder if there's another way to get an output of: Directorname\Director;Producername\Producer;Actorname\Role etc

To further darichman's never-ending quest to transform MC—kicking and screaming—into a relational database, I offer the following query. This is for putting all people associated with a movie into the standard People field—nested as Name\Career and Name\Actor\Title: Role for acting roles.

I suggest this be included as an optional query in the default configuration—for those who might prefer a nested list for people instead of a separate field for each of the five "careers." Who knows, a nested field might even be more efficient.

Quote
Any comments on the field config file?

Oops. I wasn't fast enough. Well, maybe 1.0.14? :'(

Code: [Select]
Select PEOPLE."name" || '\Actor\' || MOVIES."title" || ': ' || CAREER."role"
From MOVIES inner join CAREER on MOVIES."mid" = CAREER."mid" inner join PEOPLE on PEOPLE."pid" = CAREER."pid"
Where CAREER."ctype" = 0 and MOVIES."mid" = @id
Union
Select PEOPLE."name" || '\Director'
From MOVIES inner join CAREER on MOVIES."mid" = CAREER."mid" inner join PEOPLE on PEOPLE."pid" = CAREER."pid"
Where CAREER."ctype" = 1 and MOVIES."mid" = @id
Union
Select PEOPLE."name" || '\Screenwriter'
From MOVIES inner join CAREER on MOVIES."mid" = CAREER."mid" inner join PEOPLE on PEOPLE."pid" = CAREER."pid"
Where CAREER."ctype" = 2 and MOVIES."mid" = @id
Union
Select PEOPLE."name" || '\Composer'
From MOVIES inner join CAREER on MOVIES."mid" = CAREER."mid" inner join PEOPLE on PEOPLE."pid" = CAREER."pid"
Where CAREER."ctype" = 3 and MOVIES."mid" = @id
Union
Select PEOPLE."name" || '\Producer'
From MOVIES inner join CAREER on MOVIES."mid" = CAREER."mid" inner join PEOPLE on PEOPLE."pid" = CAREER."pid"
Where CAREER."ctype" = 4 and MOVIES."mid" = @id
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 25, 2009, 07:40:12 pm
o Feature: New checkbox "Ignore". When selected all fields marked Ignore are displayed, otherwise, none marked ignore are displayed.

This is nice.

Quote
o Changed: Updated field configuration to contain Darichman's suggestions too.

This is not. Not that there's anything wrong with darichman's suggestions, but most of them are dependent upon his personal PVD configuration. I have some of the same in mine, and I excluded them from my proposed default configuration because I realized they would just create unnecessary confusion. Users are going to try these queries, find they don't work, and likely fail to recognize it's because they have not created exactly the same custom field in PVD and/or configured a plugin or script to put the same data in that field.

The issue is not restricted to custom fields. To illustrate, darichman and I use both IMDb and AllMovie as data sources. I believe darichman's approach is to populate the PVD standard fields with IMDb data, and to put all of the AllMovie data into custom fields. My approach is to blend the two together, using very deliberate field overwrite settings to do so. Fields are not necessarily used for their original (IMDb based) design purpose. For example, AllMovie Synopsis goes into Description, Tones goes into Tags. Only truly unique, additional AllMovie data goes into custom fields.

My point is, if darichman and I were new users, including these additional queries would not help either of us. All any user needs, regardless of how they do things, is the ability to easily identify the query needed to import any particular field. Standard fields should always be referred to by their proper names, regardless of what they are used for. Custom fields are accessed according to the custom display names the user has given them. Any attempt to guess what data the user has configured PVD to save in a particular field is not useful and only likely to lead to confusion and errors.

I think the default field configuration should be as follows:

1. Include all PVD standard fields that are likely to be used. Some are too trivial to bother. Others might be concatenations—like my examples for summarized media information. Present them in logical groupings of similar data.

2. Use some discretion and mark some of the fields users are least likely to use as "Ignored." The combination of the Ignore feature and the logical grouping of the fields works very well for configuring the level of detail desired. Including a bunch of fields that don't exist obscures this.

3. Include a few alternative queries. For example, nested Actors\Roles instead of Actors, nested People\Career instead of (or in addition to) a separate field for every career.

4. At the end of the list, include queries to illustrate the form required for each of the four custom field data types:

     Data Type          PVD Field Type          MC Field Type     
     SHORTSTRShort text, Select
or Multiselect list
String, List
     LONGSTR     Memo     String     
     INTNumberInteger
     REALReal number,
Rating
Decimal, Rating

The last line of each of these queries is the only one that needs to be modified. Accordingly, the default could be...
CUSTOM_FIELDS_MOVIES."dispname" = 'your PVD custom field name'
...with the instruction, "This query is for [PVD field type]. Enter your custom field name ('dispname') in the last line of the query."

I'm sorry to be belabouring a point which must seem mundane, but my experience in supporting PVD suggests it will be very important not to confuse the task of configuring the plugin with matters of PVD configuration. Once PVD is configured to produce the results the user wants, configuring the plugin should be straightforward. It's important users understand that, or they will be asking how to configure the plugin in situations where they should be asking how to configure PVD.

Quote
What about resolving it as follows: Put "Ignore" on Name and a warning in the field Description?

I see you've done this, but I assumed there would be another query for a custom Title field. It seems to me not importing the title at all is even worse than the situation you were concerned about. Now PVD can get a different title from the filename (or it will be "corrected" by the download source) and get information for a different movie. This will be imported by the plugin with no indication whatsoever that the error has occurred. It would have been better to replace Name with the incorrect title. At least then the meta data would be consistent, but for the wrong media file. ;)
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 26, 2009, 02:40:37 am
I think the default field configuration should be as follows:

Should I revert to the previous version of the file, then? I haven't used PVD custom stuff, that's the reason why I ask all the time if you guys have tested the file!

If not, could you do me a favor and trim the file?

Please leave the original order and field names so I can use a diff tool to see the changes...
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 26, 2009, 05:57:57 am
How about this one.

- Updated Original Language query
- Date uses Date (Year)
- Removed custom fields Released by, Production Credits, AMG Category, AMG Type, custom genre fields, attributes, themes, tones, moods, flags, colour type, book, set in and review
- Mapped "Tags" to "Keywords" in MC
- Added in standard "Rating" and "Other Rating"
- Added in the People concatenated expression at the bottom, ignored by default
- Added in each of the four custom queries at the bottom, ignored by default (user replaces "FIELDNAMEHERE" with their field name)

As far as I can tell, all the default PVD fields are there, with no custom ones included (apart from the templates)
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 26, 2009, 12:15:58 pm
Ok, can we agree on this version of the config file (http://www.tonsbergibk.com/PvdImportFieldConfig.xml)?
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 26, 2009, 03:53:23 pm
Quote
Ok, can we agree on this version of the config file?

I've only did a quick comparison between darichman's proposal and this one, and decided to trust you reflected his changes appropriately.

Here's a list of the changes I made. The resulting configuration follows.

1. It probably doesn't matter, but I've deleted Sep=";" for fields types other than list.

2. .Title: Changed type to "User string" and deleted comment.

3. Release: Changed name to ".Release Date, type to "User string" and added comment "Use expression to convert format to YYYY-MM-DD for correct sorting."

4. Series: The query is a little different than mine, but looks okay—I'll test it later.

5. .Release: Added as ignored field with comment "A standard field in PVD which is not currently used and therefore available for custom use."

6. People: Changed comment to "Nested field alternative to using separate fields for actors, director, screenwriter, composer and producer."

7. .Rating User: Changed "," to "." in field name and changed McField to "User decimal."

8. .Rating Other: Added "McField: User decimal or integer / Comment: Field type depends on source."

9. .Budget and .Box Office: Changed McField to "User integer."

10. .MediaInfo Video/Audio: Changed comment to "Summary of media file video/audio information."

11. Custom field examples: Changed comments to "This query is for [PVD field type]. Enter your custom field name ('dispname') in the last line of the query."

The revised configuration file is posted here (http://www.videodb.info/forum_en/index.php?topic=1318.msg5223#msg5223). BTW, why are we unable to attach such things here?! >:(
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 26, 2009, 04:17:25 pm
I've only did a quick comparison between darichman's proposal and this one, and decided to trust you reflected his changes appropriately.
Well, the file seems to converge towards something now anyways :)

Can you post a note when you've tested "Series"? I think we can call the package sort of healthy then...
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 26, 2009, 07:33:01 pm
What is the expected behaviour for "Series"? (I don't use it so had to make some dummy files with episodes etc)

I just tested it and the current expression is successful in pulling in the name of the episode from PVD.

IE
TV Show Name
 > 1. Episode Title

Code: [Select]
Select MOVIES1."title"
From MOVIES inner join EPISODES on MOVIES."epid" = EPISODES."epid" inner join MOVIES MOVIES1 on EPISODES."mid" = MOVIES1."mid"
Where MOVIES."mid"=@id

Will pull "Episode Title". I'm not sure that's what we want... It goes in the [.Title] field anyway...

Does it serve any other purpose? Cut it?


Otherwise... I just ran the plugin on all my movies (~600) using mass import. No issues noted.
Are we ready to push this out the door?
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 26, 2009, 08:56:44 pm
Quote
What is the expected behaviour for "Series"?

The expected behaviour is to return the name of the series for an episode record. The query in the default was incorrect—it should be as follows:

Code: [Select]
Select MOVIES1."title"
From MOVIES inner join EPISODES on MOVIES."epid" = EPISODES."epid" inner join MOVIES MOVIES1 on EPISODES."mid" = MOVIES1."mid"
Where MOVIES."mid"=@id

What puzzles me, is this looks the same as the query you've posted. I though I was going to check this because it was different than mine. ?

I have a meeting. Will check more later...
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 27, 2009, 02:12:27 am
Quote
What puzzles me, is this looks the same as the query you've posted.

I suppose this is what I get when I formulate queries by trial & error, and never really understand why it works.  :P

In my testing, using Select MOVIES1."title" returns the series title, and Select MOVIES."title" the episode title. Are you sure you were using the former?
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 27, 2009, 02:48:43 am
Quote
Can you post a note when you've tested "Series"? I think we can call the package sort of healthy then...

My testing indicates the series query works. I just don't know why. ;)

I thought it would be a good idea if the default included more of the standard PVD fields. So I added the following—all as "ignored." I think the resulting "coverage" is pretty good. It doesn't include everything, but those left out are rather obscure or of questionable relevance in MC.

Rating
.Rating Name (name associated with .Rating Other)

Statistics
.Date Added
.Date Modified
.Bookmark (so items bookmarked in PVD can be flagged in MC)
.Loaned

Media
.Media Index
.Media Barcode
.Media Location
.Media Quality
.Media Features

The updated configuration file is here (http://www.videodb.info/forum_en/index.php?topic=1318.msg5230#msg5230).

Quote
Are we ready to push this out the door?

I think so, but for one thing. PVD 0.9.9.7 has just been released. I think we should make sure the queries work in it before releasing. I doubt the queries are affected at all, but better safe than sorry. Also, as long as we're sure 0.9.9.7 is stable, it would be preferable to refer new plugin users to it, rather than 0.9.9.6. I recommend waiting a few days at least. Who knows? Maybe a Theater View upgrade (with a video information panel) will materialize by then. ;D
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 27, 2009, 07:37:11 am
Rick - I don't use series, so maybe I hadn't set something up right when I was testing it? Who knows. I'll trust your judgement :D

I have just installed PVD 0.9.9.7 and set it as the database for 1.0.13 ... It appears to work fine with Rick's new config file (note that my custom fields in MC are named differently to the plugin, so these ones were tested only by looking at the SQL result column - Otherwise mass import and a "Send To" of 100 files worked perfectly)

I think we have a winner.

Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 27, 2009, 08:36:45 am
I have also installed 0.9.9.7 and done mass imports + "Send to" . It seems as if this version of PVD works just fine. Another plus is that there is a proper installer associated with the release.

So where do I post the released version? in the "Media Center 13" forum?
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 27, 2009, 01:30:39 pm
Rick - I don't use series, so maybe I hadn't set something up right when I was testing it? Who knows. I'll trust your judgement :D

That's nice. I plan to listen carefully for uproarious laughter from those who actually understand SQL. Our only defence is, "It works, doesn't it?"   ;D

I had difficulty wrapping my mind around this one. An episode is just another record in the MOVIE table, so the query has to match the "parent" series record in the same table to get the name of the series. I do hope I've got it right—episode records that don't include the series they are part of are rather useless.
Title: Re: Personal Video Database Import plugin - Test log
Post by: rick.ca on May 27, 2009, 02:11:29 pm
I have also installed 0.9.9.7 and done mass imports + "Send to" . It seems as if this version of PVD works just fine. Another plus is that there is a proper installer associated with the release.

Yes, I'm sure none of the changes affected the database. If anything, it's probably more stable than 0.9.9.6. And, yes, the installer is a big plus.

The development bar has been raised a little, however. I think your going to have to add Twitter functionality—so users can tweet what they're doing with the plugin. ;)

Quote
So where do I post the released version? in the "Media Center 13" forum?

It looks like Third Party Plug-ins, Programs, and Skins (http://yabb.jriver.com/interact/index.php?board=5.0) is the place for it. Judging from the size/popularity of some topics there, J River doesn't offer child boards for such things—which is the only practical way to have different topics for releases, support, feature suggestions, etc. If it seems necessary, we could probably set that up at the PVD site, but I'm not sure redirecting MC users there is a good idea.

You should post an "announcement" in the "Media Center 13" forum, with a link to the release topic.
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 27, 2009, 08:02:08 pm
Yes, post an announcement on the main MC13 board (feel free to use some of my screens if you like) and the main release thread on the Third Party board. We could see how big it gets and decide whether or not to split to separate threads later maybe?

Good luck :)
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 28, 2009, 01:31:10 am
Darichman, can you please post a new screen dump with all the new GUI on it?

Thanks..
Title: Re: Personal Video Database Import plugin - Test log
Post by: darichman on May 28, 2009, 03:51:41 am
Here are a few of PVD, the plugin and MC with imported data.
Was there anything else specifically you wanted to illustrate?

Piccies (http://www.pix01.com/gallery/7E547B99-5004-478D-A4AC-37AD672F51A4/PvdImport_Gallery/)
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 28, 2009, 05:35:56 am
No, that's fine. I just wanted to post a picture from the plugin. I'll also post a pic from one of the views you uploaded.
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 28, 2009, 05:38:39 am
Btw, regarding v14: I can't see any reason why PvdImport shouldn't run on V14. The next step would be to debug with v14 installed, but that'll have to wait I guess..
Title: Re: Personal Video Database Import plugin - Test log
Post by: Two Wire on May 28, 2009, 10:04:19 pm
Raldo, darichman, and rick.ca, thanks for all your efforts in developing this outstanding plugin. What a time saver. I have been following this thread and trying each release. I have been doing this quietly beause any coment I had was already discussed by you three, and I know very little about plugin development.
However, having manually filled database fields for more videos than I care to admit (I would be embarrased if I had to give an accurate account), it has given me my lifeback. This is good enough to ask for donations.   ;D
Looking foward to future developments.  

It would be nice if JRiver had a currency format option for related fields. 


Thanks again!
Title: Re: Personal Video Database Import plugin - Test log
Post by: raldo on May 29, 2009, 10:13:11 am
This is good enough to ask for donations.   ;D
Looking foward to future developments.  

I never thought of that :)

It would be nice if JRiver had a currency format option for related fields.  
There are some formatting options in Firebird SQL.

Could you please post in the  Third Party Plug-ins, Programs, and Skins (http://yabb.jriver.com/interact/index.php?topic=52145.0)  thread ?

I was hoping to kill off the current thread, it being the "test thread"...