Catalog corruption!

[Andriy.Okhrimets]

Nice feature request. Send it to PO, currently software do not manage backups it just creates them. But catalog files are relatively small(it is actually small database), so I have no problem keeping them forever.
It keeps only last catalog backup inside catalog folder as I recall.

[harald_walker]

But catalog files are relatively small(it is actually small database), so I have no problem keeping them forever.

I have multiple catalogs and within each catalog the SQLite database file is between 100 and 250 MB. Catalog are all on the local SSD disk and space is limited there. Each GB counts. I was able to regain several gigabytes of disk space by cleaning up old CaptureOne backups. For incremental backups I have my time machine backups.
If C1 would keep 1 or two copies, that would be more than enough and is basically just for disaster recovery like Aarons problem.

[Andriy.Okhrimets]

A small hack here I always specify backup catalog location folder to NAS (do you know that you can change that) and store them forever. )

[Paul Lindqvist]

I have multiple catalogs and within each catalog the SQLite database file is between 100 and 250 MB. Catalog are all on the local SSD disk and space is limited there. Each GB counts. I was able to regain several gigabytes of disk space by cleaning up old CaptureOne backups. For incremental backups I have my time machine backups.
If C1 would keep 1 or two copies, that would be more than enough and is basically just for disaster recovery like Aarons problem.

You don't have to have the back up on your local SSD, just set it to be stored somewhere else. As for backing up the DB files, there a plenty of software that can manages that and do it well.

[harald_walker]

I have multiple catalogs and within each catalog the SQLite database file is between 100 and 250 MB. Catalog are all on the local SSD disk and space is limited there. Each GB counts. I was able to regain several gigabytes of disk space by cleaning up old CaptureOne backups. For incremental backups I have my time machine backups.
If C1 would keep 1 or two copies, that would be more than enough and is basically just for disaster recovery like Aarons problem.

You don't have to have the back up on your local SSD, just set it to be stored somewhere else. As for backing up the DB files, there a plenty of software that can manages that and do it well.

Please read again what I first wrote, it also makes the same backup within the catalog file on the SSD. I know that I can change the backup location in preferences to an external disk.

[Paul Lindqvist]

I have multiple catalogs and within each catalog the SQLite database file is between 100 and 250 MB. Catalog are all on the local SSD disk and space is limited there. Each GB counts. I was able to regain several gigabytes of disk space by cleaning up old CaptureOne backups. For incremental backups I have my time machine backups.
If C1 would keep 1 or two copies, that would be more than enough and is basically just for disaster recovery like Aarons problem.

You don't have to have the back up on your local SSD, just set it to be stored somewhere else. As for backing up the DB files, there a plenty of software that can manages that and do it well.

Please read again what I first wrote, it also makes the same backup within the catalog file on the SSD. I know that I can change the backup location in preferences to an external disk.

Im not sure i understand, are you saying that it backups the database on two locations, one always in the catalog folder and one in the location you specified in the preference panel ?

I just tested this and i have no backup databases in the original catalog folder, just get them in the location i specified.

[harald_walker]

Im not sure i understand, are you saying that it backups the database on two locations, one always in the catalog folder and one in the location you specified in the preference panel ?

I just tested this and i have no backup databases in the original catalog folder, just get them in the location i specified.

Yes, exactly that. Here is a screenshot.



I guess this is another bug then.

[Paul Lindqvist]

Im not sure i understand, are you saying that it backups the database on two locations, one always in the catalog folder and one in the location you specified in the preference panel ?

I just tested this and i have no backup databases in the original catalog folder, just get them in the location i specified.

Yes, exactly that. Here is a screenshot.



I guess this is another bug then.

Strange, could you please post a screenshot of the location where you specified the backup's to be stored ? Does the timestamps and size correlate ?

I tested this and it does not happen on my installation.

[Eric Nepean]

One thing I have observed with Capture One Pro is that faced with a big job, it may become non-responsive for minutes or even more than an hour. I have learned that if you interrupt its work by killing it, you will loose something. There are times when it crashes, this it announces with an error message, and then you have lost some work also.

When COP becomes non-responsive, I launch the activity monitor, and check how how much % CPU and Total Memory (GB) it is consuming. If it is non-responsive and see it consuming less than 1% CPU (which happens to other SW but not in my memory to COP) then I would know it has become lost and I must reset it. But more commonly I see it consuming a high % of CPU, and the amount of RAM may rise to many GB. Once the virtual RAM consumed is is many GB larger than the physical memory, then I know there is a very big job - I have seen COP crash when it has consumed over 60GB of RAM on my 16GB machine (the extra 45GB is virtual RAM) - and then you get an error message.

But if the CPU is still being consumed, and the the Total Memory is not too much larger than yopur installed RAM, then just let it keep running. On a large import I have seen it be unresponsive for 75 minutes.

I must say that being unresponsive for such a long time with no indication to the user is a poor user interface design. The processing time is a limitation that must be acceptedas it is a big challenge to reduce and is affected by the users HW, but the non-responsive part is simply a poor design choice IMO.

[NN635115164314618060UL]

I have exactly the same issue as Harald describes - my main catalog on an SSD drive had grown to over 25Gb in size (all referenced images!) but after crashing due to the memory leak in CO 9.0, became corrupted. In trying to restore from a recent backup, I discovered that the catalog package contained over 28 backups taken over the course of the last 12 months IN ADDITION to the backups I had stored on a separate drive.

One by one over the next few days, other libraries and several backup copies became corrupt after crashing. In 9.0.1 things have improved but the crashing still happens far too regularly. Trying to restore or import from a backup, C1 is unresponsive for long periods and consumes vast amounts of memory. I tried leaving it overnight to import an backup library into a new one which it said would take 12 hours. Checking on it an hour or so later, I was informed my system had run out of application memory and my boot drive had run out of space.

I've got just 11 days left of my trial - how on earth can you properly test this out when it doesn't seem to run for more than half an hour before it crashes? Shame because I really like what I've seen of the new features but it is slowly driving me nuts!

[CaptureMann]

Poor Aaron just wants a solution to his problem - not a bunch comments about why he did or did not back up his catalog.
Shame on all you perfect people for trying to malign him instead of helping him with his problem.

I agree - released software should have extra safeguards in it to prevent data loss. I've used both Aperture and Lightroom for years with approx 30-35k images and never had such a problem.
Data loss is the one thing a Digital Asset Manager should never cause.

Now I have the same problem Aaron does.
In my case, I was simply adding a keyword when Capture One hung and the fans on my machine cranked up.
Activity Monitor showed Capture One using between 500 and 600% of the CPU.

Now I've lost my entire catalog...
Not such a big deal for me - this is day 2 of my 30 day trial and I just imported a database from another program.
But still - not confidence inspiring.

I can't make hierarchical keywords work on my DB import - but I'll ask that question in another topic since this one is on corruption.

Cheers!

[Jimmy D Uptain]

Poor Aaron just wants a solution to his problem - not a bunch comments about why he did or did not back up his catalog.
Shame on all you perfect people for trying to malign him instead of helping him with his problem.

This!

There is a flaw in the system, period!!!!

My catalog was backed up, and guess what, the backup was corrupt as well. I ended up having to rebuild the catalog. This took a couple of days.
I have been a C1 user for quite a while, and while the RAW conversion has taken huge leaps, the catalog leaves much to be desired.

[NN635465378066978588UL]

I am using C1 Pro for Sony 9.0.1 on an iMac with OSX 10.11.2. I have now had multiple catalogue corruptions (my database is less that 10k images), and have tried all the suggestions used in the forum. I can still work with a slow system (much slower than C1 , but having to create a new catalogue and import all images again on a weekly basis is just too much.

I absolutely loved C1 for its workflow and results, and the addition of the DAM function made me think this is now the best of all worlds. However, given the experience since C9 came out, I am now back to a combination of LR6 (for DAM purposes) and DXO Pro10 for raw processing. In my humble view not the best solution, but at least I will be able to do my work without having to wait for previews and having to rebuild catalogues every now and then.

[HansB]

Just briefly reading into this thread. Did you try these suggestions?

Have posted this for another person who had the same problems:
I managed to "recover" a destroyed catalogue by simply creating a new one and importing the old damaged one.

Dear Phase One - with all due respect for the great work on this software. Maybe it is time to make a public announcement that Capture One 9 is at present NOT working reliably with Canon files.

A pretty serious issue that would affect one or two photographer out there.

Rebuilding is also possible from command line. I don't have a broken catalog to test it, but maybe somebody wants to try it on a copy of a broken one? It's command line, so cd into the catalog file.

Code: Select all

cd <YOUR>.cocatalog
mv <YOUR>.cocatalogdb <YOUR>.cocatalogdb.yourbak
echo .dump | sqlite3 <YOUR>.cocatalogdb.yourbak | sqlite3 <YOUR>.cocatalogdb


Another option might be to query VACUUM; on the catalog database.

Not sure if this works, but maybe it helps creating a solution.


Regards,
Hans

Regards,
Hans

[mercator]

It makes automatic backups of catalog file weekly. So in case of corruption you may revert.
Catalog files are always a single point of failure in any DAM, so back it up or sync to other drive to prevent cases such you are describing. I've got similar case with Lightroom and it cost me 2 weeks of work restore to correct state.

CO actually uses a database to store - most of - its data (basically all meta information but not the images or thumbs). And the one used is actally not bad if employed the right way, eg making use of transactions. It would be interesting to find out when CO actually considers a piece of work to be completed (eg after an individual adjustment or when you move to the next image). Depending on when such commits are done, you basically never lose anything. The most likely use case for losing data would then be a crash of your HD - even pulling the plug should actually NOT harm your database...