Mask corrupted / C1 kills layer masks?

Discussions, questions, comments and suggestions regarding Capture One PRO, Capture One PRO(For Sony), Capture One DB and Capture One Express(For Sony) 9.x

Mask corrupted / C1 kills layer masks?

Postby BeO » Thu Feb 25, 2016 12:42 pm

Hi.

If that's true then this is a severe issue.

It appears to me that during startup, on loading "all images" or other collections, maybe on other occasions too, you can "loose" layer masks!

Or, it is detected at that time that you have lost layer masks (corruted files?)

If that's true, the problem is that you do not even notice this unless you look into the application log!

And even then, the log not even contains the file for which the mask is corrupted.

Masks are being "regenerated" which seems to be nothing else than to generate an empty mask!

It happened to me at least 30 times since summer last year, and indeed I miss some layer masks (the layers are still there in the LA tool but no pixels in the mask). I concentrated my search for missing masks on images whose .comask file have the latest changed date/time in Windows explorer when I detected corrupted mask message today.

It happens with version 8 and 9. And on two different PCs.

Application log:
"Mask corrupted, regenerating"

At least the application log should issue the image names! Even better if C1 shows a message to the user. Even better if the root cause would be eliminated, I hardly can believe that this is a file system / drive issue, as I have never ever experienced a corrupted raw file for example, only layer masks...

I would be interested if you also have this error message in the application.log and/or if you also miss masks.

Thanks and cheers
BeO
Last edited by BeO on Sat Feb 27, 2016 10:23 pm, edited 1 time in total.
BeO
 
Posts: 511
Joined: Thu Nov 13, 2014 4:18 pm

Re: Mask corrupted / Loosing layer masks

Postby Michael11 » Thu Feb 25, 2016 2:30 pm

Hi,

as far as I know C1 expects the related files (*.comask) at certain places - at least in Sessions which is the mode I'm working in.

Is there a chance that C1 in your working ambient could loose connection / track to the subfolder where this information is stored (expected)?
Michael11
 
Posts: 280
Joined: Fri Aug 31, 2007 12:50 pm
Location: Brazil

Re: Mask corrupted / Loosing layer masks

Postby BeO » Thu Feb 25, 2016 4:14 pm

Thanks Michael,

No, this is not the case. Before and after such messages in the log, I have the same number of .comask files the the adjustments folder. Many of them have a new modified date and time (time of C1 startup) but all have an old creation date.

Also, I often have several layers/masks. Usually one one layer mask is lost but the others are there, I assume all masks are stored in the same .comask file for an image.

Cheers
BeO
BeO
 
Posts: 511
Joined: Thu Nov 13, 2014 4:18 pm

Re: Mask corrupted / Loosing layer masks

Postby BeO » Thu Feb 25, 2016 4:20 pm

If I knew which images are affected (as soon as they are affected / corrupted) I could copy the .comask from an older backup.

Anyone else with "mask corrupted" message in the application.log?
BeO
 
Posts: 511
Joined: Thu Nov 13, 2014 4:18 pm

Re: Mask corrupted / Loosing layer masks

Postby SFA » Fri Feb 26, 2016 10:39 am

I suspect this message may not be telling the entire story to us, the uninitiated.

That it is able to "regenerate" and on the basis that I have not yet identifies a missing mask despite a lot of use and, on occasions, a lot of copy and apply activity suggests to me that this is an internal message that something is not right with something in memory (whether just now being loaded or something from cache is unstated, and is therefore being recreated in the memory stack from the comask file. If I am right we can think of this as inbuilt process error correction.

The use of multithreading technology by both CPU and now GPU means that arts of the process are picked out processed and re-combined later. In some ways the linear process power CPUs (usually top gaming machines) from a decade or more back offered a more controllable processing opportunity than the multicore and multithreaded chips available today. However since linear processing is no longer a realistic option in a process intensive application where the usage demands a fast perceived response split and recombine is the only option. Check and correct to ensure that everything has worked as planned is therefore a requirement. I suspect that might be what is being recorded in parts of the log file in order to monitor how well the designs applied are achieving their goals.

Or something like that.

I could well be wrong but so far I have not observed a total loss of masks (or any other adjustments other than one occurrence of a partially damaged .cos file back in V7 days). On that basis I have concluded that the message line in the log file is informational rather than something that relates to a permanent error.



Grant
SFA
 
Posts: 6094
Joined: Tue Dec 20, 2011 9:32 pm

Re: Mask corrupted / Loosing layer masks

Postby BeO » Fri Feb 26, 2016 10:40 pm

Hi Grant,

I have meanwhile conducted a lot more tests. I finally ended up with a new catalog with only 1 image to ease my tests. And I think there is a very little error likelihood for my findings.

Case 1: .comask file not available

If the file is not available (deleted, renamed, not accessible) then a new (empty) file is generated. No mask pixels inside.
The layer properties (slider values) are preserverd, as they are stored in the database (ZVARIANTLAYER). In this record there is a unique ID for the layer mask (ZMASKUUID).
This ID is not changed, i.e. if you find the .comask file in one of your Adjustment folder backups you can replace the newly created empty file with the old one and everything is fine.

In addition to "mask corrupted, regenerating" you also get a line in the application.log saying "file abc.comask not found", so you know which variant is affected.

This is true for v9.0.3 not tested for earlier versions.


Case 2: File exists but is "corrupted".

I have .comask files in my backup which were stored on a Synology NAS, don't know if this is the (or the only) reason for this or similar problems, as most of such files work fine.

However, C1 does not regonize some files as valid and says "mask corrupted, regenerating". But no mention about the affected file!

And in this case, the UUID in the database is changed!

It seems that the corrupted (or unidentified) layers are replaced, the layers' mask UUID is newly generated!

That means your backup .comask files (with the original UUIDs) are totally useless, you cannot replace the file to regain your mask.

Useless unless you also search for the old UUID in the old catalog and replace the new UUID in the new catalog with a database tool. Which I did and which proves that the neither the old backup file nor the mask in it are actually corrupted.


What they did half-way right for a missing file (right: don't change the UUID, log the filename. wrong: no notice of the user in the application GUI), they did not make it right for single layer masks failing (wrong: change UUID, no filename in the log, no notice of the user in the application GUI).

------

I have 30 such lines in the log since last summer, randomly distributed. I guess that an earlier version of C1 (from last year) re-generated the UUID for some layers for some unknown reason, I don't really believe that my .comask files got really corrupted.

I lost another 10 masks just yesterday on the new computer, with my optimized, exported and verified catalog.

To prevent further mask issues I think I have to go back again to an (not to) old catalog and try to find another way to migrate.

Cheers,
BeO
BeO
 
Posts: 511
Joined: Thu Nov 13, 2014 4:18 pm

Re: Mask corrupted / C1 kills layer masks?

Postby BeO » Sat Feb 27, 2016 9:35 pm

I repeated the migration procedure (copy catalog and adjustments from old notebook to new PC) but this time copied all files with xcopy /v via a external drive to be sure the files get not corrupted in the copy process.

Cache files were then generated on starting C1.

Then starting / closing C1 a few times. Loading catalog, all images. Navigating in the catalog. No edits.

1. Roughly 300 .comask files then have gotten a new modification date in windows explorer. Why? This happens every time I load all images.

2. From those 300, 50 comask files have been updated in the database (i.e. in ZVARIANTLAYER they kept their ZMASKUUID BUT they got a new primary key (Z_PK)). Why?

3. So far so good (?), BUT 2 variants of those 50 lost layer masks (1 image lost all 9 masks, the other image lost 1 of several masks)!

So, Why does C1 modify the 300 files on loading all images at all?
Why does C1 even change the primary key for the 50 layer masks?
Isn't both itself a problem, at least beard the risk of failures/corrupts?

Why got 2 files corrupted masks?

I am sending two files (Oringial comask file and corrupted comask file which lost one mask) to my support case.

Cheers
BeO

P.S. I just started C1 again to confirm step 1, I then again lost another mask !!!!
BeO
 
Posts: 511
Joined: Thu Nov 13, 2014 4:18 pm

Re: Mask corrupted / C1 kills layer masks?

Postby RichardT » Sat Feb 27, 2016 11:14 pm

BeO wrote:Anyone else with "mask corrupted" message in the application.log?


Hi BeO,

I only have 410 .comask files in a catalog of 33592 referenced images.

I don't have any "mask corrupted" messages in the application.log files.

The modification times of my .comask files are not updated when I select the "All Images" collection, but only when I actually select and view an image that has a mask.

Richard
User avatar
RichardT
 
Posts: 303
Joined: Mon May 06, 2013 8:58 pm
Location: Berkshire, UK

Re: Mask corrupted / C1 kills layer masks?

Postby BeO » Sat Feb 27, 2016 11:42 pm

Richard,

Thank you!

Opening a .comask file with notepad++, it seems that some version number("ver") and "userinfo" is stored. As I am running v8 as a registered user but v9 as a trial, mabye C1 updates this info in the file.
Files are modified as I go scroll through the browser. But C1 could have done this while generating preview imo.

However, you confirm what I also see, repeatedly restarting and openening the same image modifies the file again. Which is silly imo.

I am happy that C1 does not change the raw files and eventually corrupt them!

I would attribtue this to either v9 or the new PC but have noticed this with v8 on the old notebook too, so it seems to be an issue with C1, if that's not going to be sorted out that's it for me.

First time ever I really get angry about the issues with C1 :twisted:

Thanks again Richard
BeO
BeO
 
Posts: 511
Joined: Thu Nov 13, 2014 4:18 pm

Re: Mask corrupted / C1 kills layer masks?

Postby SFA » Sat Feb 27, 2016 11:51 pm

Hang on BeO,

The New V9 engine probably does imply changes to the information in the comask files - especially interesting to deal with if you have variants using different engines all the way form V7 perhaps.

As I recall there were also some step changes in the V8 engine and I assume they might end up with the same result.

So changes are not necessarily a problem. Losses, unexplained, possibly are - although why they apparently go missing may not be so easy to confirm or identify. Indeed is anything lost? It may be possibly to have a comask file that was created, is actually doing nothing and is therefore binned on development engine upgrade since it has no purpose and cannot be upgraded.

I'm speculating of course - but it is speculation that may need to be considered.


Grant
SFA
 
Posts: 6094
Joined: Tue Dec 20, 2011 9:32 pm

Re: Mask corrupted / C1 kills layer masks?

Postby BeO » Sun Feb 28, 2016 9:46 am

Grant, thank you.

SFA wrote:Hang on BeO,

The New V9 engine probably does imply changes to the information in the comask files - especially interesting to deal with if you have variants using different engines all the way form V7 perhaps.
Grant


I just imported a new file into V8 and added layers and masks. Closed C1. Started C1. View this image again. Comask files is modified. No other application changes a file when viewed only without having made any changes, any auto-save feature should respect that.

But yes, not a problem as such, just weird.

SFA wrote: Indeed is anything lost?

Grant


Indeed masks are lost. Slider values are preserved. Believe me, if a mask has red pixels and sliders are applied, and after the message "Mask corrupted, regenerating" the comask file size is smaller, the layer sliders still there but no red mask anymore, sliders with no effect, then the original mask is lost.

Cheers
BeO
BeO
 
Posts: 511
Joined: Thu Nov 13, 2014 4:18 pm

Re: Mask corrupted / C1 kills layer masks?

Postby RichardT » Sun Feb 28, 2016 11:57 am

I've just done a few experiments in a new catalog.

I imported some RAW files and created masks on two of them. Corresponding .comask files were created.

The following tests were made without changing the actual masks on the images.

With one of the images selected, I restarted Capture One. The modification time for the selected image's .comask file was changed, but the one that had not been selected was not changed.

I then changed the selection to the other image and the modification time for its .comask file was immediately changed.

Changing the selection again did not change the modification time of either .comask file. It looks like the modification time is only changed the first time image is selected after Capture One is started.

I took copies of a .comask file before and after its modification time was changed and used the WinMerge app to compare them. The contents of the files were identical. i.e. the modification times had been updated, but the contents were the same.

I noticed another oddity. I selected an image file that did not have a mask. I cloned the image and added a mask to the clone. A .comask file was created. At this point one variant of the image had a mask and the other didn't. I then deleted the clone that had the mask. The .comask file still existed, even though the only remaining variant of the image didn't have a mask. The modification time of the this 'orphaned' .comask file did not get changed when I selected the image after restarting Capture One. Running Verify on the catalog detected no errors.

During these tests I didn't get any "mask corrupted" messages in the application.log file, or notice any masks go missing.

Richard
User avatar
RichardT
 
Posts: 303
Joined: Mon May 06, 2013 8:58 pm
Location: Berkshire, UK

Re: Mask corrupted / C1 kills layer masks?

Postby SFA » Sun Feb 28, 2016 1:32 pm

BeO, Richard,

Interesting observations.

Am I correct to assume that both of you have done the analysis working just with catalogues?

I will see if I can find anything of similar interest in my sessions. I have certain seen the "regenerating" message although form memory less so since V9. That does not mean much of course - it may depend on what I have been working with.

Grant
SFA
 
Posts: 6094
Joined: Tue Dec 20, 2011 9:32 pm

Re: Mask corrupted / C1 kills layer masks?

Postby Richard Allen » Sun Feb 28, 2016 3:34 pm

I have a situation with my masks in that if I open a catalogue a mask for any image won't show until I go to the mask, click on it and select Adjust>Fill Mask and then the mask comes back.

Is this similiar situation to your problem?

I'd also like to know why Capture One doesn't pick up the pre-created masks?

Richard
"Only two things are infinite, the universe and human stupidity, and I'm not sure about the former." -- Albert Einstein
Richard Allen
 
Posts: 269
Joined: Fri Mar 01, 2013 9:38 pm
Location: Waltham Abbey, Essex, UK

Re: Mask corrupted / C1 kills layer masks?

Postby BeO » Sun Feb 28, 2016 5:01 pm

Hi Richard,

Adjust>Fill Mask: If you mask any closed geometry, then Fill mask fills the empty pixels within that geometry. In case your mask is empty, then it fills the whole image. Is it that what you mean? When you say "then the mask comes back." does it means the whole image is masked and you expect that?

I can't imagine a use case for a mask with 100% image coverage (and equal opacity, gradients are a different story of course).

In case you usually have masks with lesser area coverage (or gradients) and you need to Fill mask and then you have 100% coverage then the mask was either empty by you or you have found a mask which was killed by C1.

10 minutes ago another image lost its mask... (v9 trial on new PC)

Maybe C1 writes bytewise new information like version of userinfo and if the new version or userinfo has more bytes then it overwrites some bytes in the file and therefore corrupts one or more masks in the file?

Yes Grant, I don't use sessions.

Cheers
BeO
BeO
 
Posts: 511
Joined: Thu Nov 13, 2014 4:18 pm

Next

Return to Capture One 9.x Software



Who is online

Users browsing this forum: No registered users and 3 guests