repeated preview generation

Discussions, questions, comments and suggestions regarding Capture One PRO, Capture One PRO For Sony / Fujifilm, Capture One for Phase One and Capture One Express For Sony / For Fujifilm 20.x for Windows

repeated preview generation

Postby atelzzz » Mon Jan 20, 2020 6:39 pm

Issue 1: Session of about 750 fuji x-t3 raw files. Each time I open it, previews are re-generated. And it is about 1 hour at about 100% CPU. Long standing issue, since CO 9 I think.... Why not fix it?

Issue 2: processing fuji x-t3 files to JPEG, about 500 images, the processing stops at some point, doing nothing. Relosution: close CO, kill imgCoreProcess using windows task manager, remove temp file on which processing hung, restart CO. After this is done, processing resumes. This use to happen in CP 12 more often. It happened in CO 20 first time to me today. I believe CO 9 and 10 did not have such issue, it was introduced in CO 11, and have been there since... Not a big deal, but annoying? Yes. Easy to fix? Yes also!
atelzzz
 
Posts: 25
Joined: Mon Jan 05, 2015 3:14 am

Re: repeated preview generation

Postby Paul_Steunebrink » Mon Jan 20, 2020 8:03 pm

Regarding issue 1, is this happening on the same computer over the years? If so, I suggest to create a new user account and run Capture One a few times from that account.

What I actually suggest - background information - is that it could be related to your computer setup and in particular the user profile.
User avatar
Paul_Steunebrink
Certified Professional
Certified Professional
 
Posts: 10072
Joined: Tue Feb 06, 2007 8:49 pm
Location: The Netherlands

Re: repeated preview generation

Postby atelzzz » Mon Jan 20, 2020 8:34 pm

Thank you. I will try that, but it will take me some time to confirm. Issue number 1 happens more often with large sessions, but not always. I just opened sessions with only 150 images, and it was all right.

You have "Verify Catalog or Session" command on the menu. This tells me that corrupted catalogs/sessions is OK to have (it is built in, corrupting a session). Once in a while they get corrupted.

Could it be that this preview regeneration happens on corrupted sessions? Some internal flag sets it off...

Would not it be more straightforward to find out why catalogs/sessions get corrupted, fix it, and remove this, very strange, I would say, command from the menu? Yes, I think that would be much cleaner...
atelzzz
 
Posts: 25
Joined: Mon Jan 05, 2015 3:14 am

Re: repeated preview generation

Postby C-M-B » Mon Jan 20, 2020 11:57 pm

atelzzz wrote:Issue 1: Session of about 750 fuji x-t3 raw files. Each time I open it, previews are re-generated. And it is about 1 hour at about 100% CPU. Long standing issue, since CO 9 I think.... Why not fix it?

Issue 2: processing fuji x-t3 files to JPEG, about 500 images, the processing stops at some point, doing nothing. Relosution: close CO, kill imgCoreProcess using windows task manager, remove temp file on which processing hung, restart CO. After this is done, processing resumes. This use to happen in CP 12 more often. It happened in CO 20 first time to me today. I believe CO 9 and 10 did not have such issue, it was introduced in CO 11, and have been there since... Not a big deal, but annoying? Yes. Easy to fix? Yes also!



Issue 1 is not an issue, it's the inability of CaptureOne programmers to get their s*** together. As you said, this has been around for many generations and with ever increasing resolutions and amounts of files (especially for professional photography), it slowly but surely will get less usable by every year.
What's happening is that CaptureOne does not regenerate the previews but it re-indexes the files and loads a chunk of the data into your RAM.
Thus the high CPU usage and high disk usage.

That is also the reason why any real work via NAS systems or other network based methods is basically impossible.

They claim it's due to the way Windows works - but it's really not. It's just a choice of the programmers to _not_ store the preview files (and index) in the sessions/catalog file itself. That's why the session file is so small and can simply be replaced by another sessions file (it basically hold no data except for "I am a session".)
C-M-B
 
Posts: 467
Joined: Thu Feb 25, 2010 4:57 pm

Re: repeated preview generation

Postby atelzzz » Tue Jan 21, 2020 12:43 am

Forgot to mention about issue #3, but since I am in the middle of it now, waiting, so I had a moment to check this post, I will add it to the list. Here it goes, issue 3:

Open session of about 500 images, then close CO right away. and it gets into long long "Saving ...", and it is saving something for about 30 seconds, or maybe a minute. But I have not changed anything, what is it saving? I just opened, and then closed a session. This kind of bugs, they don't do even in kinder-garden now..
atelzzz
 
Posts: 25
Joined: Mon Jan 05, 2015 3:14 am

Re: repeated preview generation

Postby SFA » Tue Jan 21, 2020 3:49 am

If you want Previews stored in a database then use a catalogue.

If you want easy flexibility and shareability use a session.

In either case when you open a catalogue or session in C1 it will attempt to load as much information as possible into whatever memory is available.

You may appear to take a performance hit up front for potentially faster editing of a set of images once they are loaded.

During the load C1 will also add the edit instructions for the variants selected as well as updating filter counters and creating default edit files (New RAW files may have been added to folders and need to be made visible via conversion, preview files and thumbnails created. etc.) All in all there can be quite some work to do.

Normally this will happen in a way that mostly reads in files and performs calculations in memory any results that might be written back to disk being saved as work continues. This will also take into account any changes in the main system settings for Preview size, and for display will add to the thumbnail and preview DISPLAYS what ever settings might apply for any Output Recipe and Proofing Profile that might be active when the catalogue or session is opened.

If you abort this process the background processing assessing whether or not there is anything that needs to be saved may be some way behind the load process. The "Saving" message is about C1 tidying up the work in memory and doing whatever housekeeping it needs to do.

If you are using a slower disk or NAS or a network connection is slow you may see this "saving" message more often, even when not aborting during opening and initial load, simply because the activity can be slower and the in-memory processing speed can get ahead of the save processes ability to keep up where there are system bottlenecks to deal with and potentially large amount of data to write.

Typically the "saving" message only appears briefly or not at all but on abort during load there may be a lot more work to do and so it may be on screen for quite some time.

There are other aspects of saving that I have not listed here but the principle is the same so the details should not be significant.

If dealing with catalogues there may also be a "Save on Close" setting to consider.

HTH.


Grant
SFA
 
Posts: 7725
Joined: Tue Dec 20, 2011 9:32 pm

Re: repeated preview generation

Postby Paul_Steunebrink » Tue Jan 21, 2020 4:35 pm

atelzzz wrote:...
Would not it be more straightforward to find out why catalogs/sessions get corrupted, ...

Could it be that your session(s) is not on the internal drive but on a network drive?
User avatar
Paul_Steunebrink
Certified Professional
Certified Professional
 
Posts: 10072
Joined: Tue Feb 06, 2007 8:49 pm
Location: The Netherlands

Re: repeated preview generation

Postby atelzzz » Thu Jan 23, 2020 12:05 am

In response to SFA. Thank you, good information. But still, if no changes have been made by the end user, it should be safe to close an app quickly. If opening a session includes writing to files (indexes etc.) that does not sound right. Opening must be strictly read operation, so that the app can be closed quickly with a single instruction, no cleanup, no data loss.

Remember how long it took for old versions of Windows to startup and shutdown. Now it is much faster. I think that they cut down on cleanup code before shutting down, made it possible to just "exit()" any time :)
atelzzz
 
Posts: 25
Joined: Mon Jan 05, 2015 3:14 am

Re: repeated preview generation

Postby atelzzz » Thu Jan 23, 2020 12:27 am

Paul_Steunebrink wrote:
atelzzz wrote:...
Would not it be more straightforward to find out why catalogs/sessions get corrupted, ...

Could it be that your session(s) is not on the internal drive but on a network drive?


Network drive failure is not of any concern to CO. There are other programs/infrastructures to deal with such failures. Let CO work on good sessions/catalogs, and fail (give an exception) on corrupted ones. That's fine with me.
atelzzz
 
Posts: 25
Joined: Mon Jan 05, 2015 3:14 am

Re: repeated preview generation

Postby Paul_Steunebrink » Sat Jan 25, 2020 9:33 pm

atelzzz wrote:
Paul_Steunebrink wrote:
atelzzz wrote:...
Would not it be more straightforward to find out why catalogs/sessions get corrupted, ...

Could it be that your session(s) is not on the internal drive but on a network drive?


Network drive failure is not of any concern to CO. There are other programs/infrastructures to deal with such failures. Let CO work on good sessions/catalogs, and fail (give an exception) on corrupted ones. That's fine with me.

Sorry I totally failed to understand what you replied.

My point is those network connections, and sometimes external drives too, have latency issues and the database in use with Capture One sessions and catalogs are particularly suspectable to that.
User avatar
Paul_Steunebrink
Certified Professional
Certified Professional
 
Posts: 10072
Joined: Tue Feb 06, 2007 8:49 pm
Location: The Netherlands

Re: repeated preview generation

Postby Wesley » Sun Jan 26, 2020 3:11 am

atelzzz wrote:Issue 1: Session of about 750 fuji x-t3 raw files. Each time I open it, previews are re-generated. And it is about 1 hour at about 100% CPU. Long standing issue, since CO 9 I think.... Why not fix it?

Issue 2: processing fuji x-t3 files to JPEG, about 500 images, the processing stops at some point, doing nothing. Relosution: close CO, kill imgCoreProcess using windows task manager, remove temp file on which processing hung, restart CO. After this is done, processing resumes. This use to happen in CP 12 more often. It happened in CO 20 first time to me today. I believe CO 9 and 10 did not have such issue, it was introduced in CO 11, and have been there since... Not a big deal, but annoying? Yes. Easy to fix? Yes also!

1 hour is excessive. It shouldn't be generating previews every time you open it.

It took me 5 seconds for C1 to load up and than 4 seconds for a session with 800 photos to load everything.
Wesley
 
Posts: 232
Joined: Wed Apr 01, 2015 11:00 pm
Location: Atlanta, GA, USA

Re: repeated preview generation

Postby atelzzz » Wed Jan 29, 2020 2:17 am

Paul_Steunebrink wrote:Sorry I totally failed to understand what you replied.

My point is those network connections, and sometimes external drives too, have latency issues and the database in use with Capture One sessions and catalogs are particularly suspectable to that.


Now I got it, you were asking.. Sorry. No, everything is on my local drive. I also have less than perfect GPU configuration, but that should not matter. I have Intel Graphics, and NVidia 1050 with 2Gb of dedicated, and 8Gb shared. Somewhere in your documentation I read that if multiple GPUs are used, they must be t he same kind and model.

Just now I opened Activities window, and it contained a progress bar, saying "Setting up hardware acceleration..." It was doing it for about 1 minute, CPU at 40%, NVidia at 15%, disk at 0%, Intel Graphics was not used. What was it actually doing for the whole 1 minute? Just curious... I updated to 20.0.2 just now. Hope it will be better. Thank you.
atelzzz
 
Posts: 25
Joined: Mon Jan 05, 2015 3:14 am

Re: repeated preview generation

Postby atelzzz » Wed Jan 29, 2020 2:28 am

Wesley wrote:It took me 5 seconds for C1 to load up and than 4 seconds for a session with 800 photos to load everything.


Sometimes it does that, opens session quickly, and I am very happy, but most of t he times it does not. It could be corrupted session..

Just out of curiosity I ran Verify session command on this session, and it showed "Checking stack image relationships" test failed. Now I will run repair, and reopen it, and see what happens...

Repair went well, and the session re-opened pretty fast without long background jobs running...

But what is this stack image relationships? How to avoid this type of session corruption? It was a new session, to which I added images, applied corrections and exported them. This I do to all my sessions, I do not keep them, after images have been exported, I removed the session, and keep only the images. So, all My sessions are fresh. Why it is on t he second or third opening I have them corrupted like this?

It would be nice to have a windows context menu "Verify Session/Catalog" command. Just right-click on a cosessiondb file, and verify/fix it. What happens now is that to verify session A, one have to open session B first (the one that was open in CO last time) and the whole UI with it, and if that session B is bad, a long background job starts... Is there a way to open CO without opening session or catalog? I could not find it!
atelzzz
 
Posts: 25
Joined: Mon Jan 05, 2015 3:14 am

Re: repeated preview generation

Postby Wesley » Wed Jan 29, 2020 7:31 am

atelzzz wrote:
Paul_Steunebrink wrote:Sorry I totally failed to understand what you replied.

My point is those network connections, and sometimes external drives too, have latency issues and the database in use with Capture One sessions and catalogs are particularly suspectable to that.


Now I got it, you were asking.. Sorry. No, everything is on my local drive. I also have less than perfect GPU configuration, but that should not matter. I have Intel Graphics, and NVidia 1050 with 2Gb of dedicated, and 8Gb shared. Somewhere in your documentation I read that if multiple GPUs are used, they must be t he same kind and model.

Just now I opened Activities window, and it contained a progress bar, saying "Setting up hardware acceleration..." It was doing it for about 1 minute, CPU at 40%, NVidia at 15%, disk at 0%, Intel Graphics was not used. What was it actually doing for the whole 1 minute? Just curious... I updated to 20.0.2 just now. Hope it will be better. Thank you.

Nvidia 1050 good enough for photo editing. Anything higher than a 1060 the performance has diminishing returns.

Capture One will see what GPU's you have and than benchmark it for hardware acceleration when you update software or the GPU driver.

atelzzz wrote:
Wesley wrote:It took me 5 seconds for C1 to load up and than 4 seconds for a session with 800 photos to load everything.


Sometimes it does that, opens session quickly, and I am very happy, but most of t he times it does not. It could be corrupted session..

Just out of curiosity I ran Verify session command on this session, and it showed "Checking stack image relationships" test failed. Now I will run repair, and reopen it, and see what happens...

Repair went well, and the session re-opened pretty fast without long background jobs running...

But what is this stack image relationships? How to avoid this type of session corruption? It was a new session, to which I added images, applied corrections and exported them. This I do to all my sessions, I do not keep them, after images have been exported, I removed the session, and keep only the images. So, all My sessions are fresh. Why it is on t he second or third opening I have them corrupted like this?

It would be nice to have a windows context menu "Verify Session/Catalog" command. Just right-click on a cosessiondb file, and verify/fix it. What happens now is that to verify session A, one have to open session B first (the one that was open in CO last time) and the whole UI with it, and if that session B is bad, a long background job starts... Is there a way to open CO without opening session or catalog? I could not find it!

I haven't had corruption problem before so I don't know about the stack image relationship.
I'm only aware of opening CO with a new, empty session/catalog. Hold shift while double clicking CO shortcut with the software closed.
Wesley
 
Posts: 232
Joined: Wed Apr 01, 2015 11:00 pm
Location: Atlanta, GA, USA


Return to Capture One 20.x Software for Windows



Who is online

Users browsing this forum: No registered users and 1 guest