Can I do a text search for "Is Not Blank"

General discussion regarding photography practices and Capture One workflow as well as integration with third party applications.
Please DO NOT post to this thread regarding OS specific concerns or questions.
Forum rules
For the sake of being thorough, please remember to note if you are using Mac or Windows.
Hopefully this will keep any confusion to a minimum.

Re: Can I do a text search for "Is Not Blank"

Postby SFA » Sun Jan 19, 2020 12:18 am

thomaskyhn wrote:
SFA wrote:
Regex, for those that know it and understand the subtly different implementations that are in use, might be nice but I think that potential users, in Capture One world, might be a very small minority and usage rather rare.

Regular expressions are used in many kinds applications; I shouldn't think that Capture One users would be less familiar with them than other software users.


It sounds like you are quite familiar with REGEX Thomas.

I do not know with any degree of certainty but think it is probable that relatively few C1 users are familiar with REGEX - perhaps especially those who benefit from hand-holding quite frequently.

I work with a business analytical system that introduced REGEX about 4 maybe 5 years ago. Despite some quite comprehensive existing facilities some influential people at one or two major clients persuaded then to do the work and include REGEX abilities

Judging by observations in their forum and comments from a few staffers I know there were a few experienced users of the application who decided they ought to find out what REGEX was and a couple of younger members of the forum who had previous exposure to the use of REGEX and that was about it.

An experienced colleague of mine who at the time was responsible for about 100 users of the application working in a large organisation full of very smart people argued against its inclusion offering the right sort of benefits to typical users on the basis that none of his users currently used REGEX in any form nor understood what it was. He suggested that, in its raw form, it would most likely cause more support issues (or just unrecognised errors) from users who might try to use it occasionally and would therefore not be beneficial. He felt there should be better option offered for a product that was not intended to be something that required an IT support operation of skill in order for it to be successfully deployed.

I think he had a point.

The front end of a search engine in a product aimed at the general public is not really the place to expect users to have familiarity with REGEX in order to be able to find a couple of photos.

For example for the original search requested in this thread the chances are that most people's use would involve words that, in English, would contain at least one of the letters a,e,i,o,u and possibly a space.

So a saved preset search that was defined as

Content Description > contains > a
Content Description > contains > e
Content Description > contains > i
Content Description > contains > o
Content Description > contains > u

and another entry for "space"

using a Match set to "ANY"

would be expected to find all images with text in the Content Description field.

I feel fairly certain that similar approaches could be taken for the other supported languages.

What are your thoughts?
SFA
 
Posts: 7720
Joined: Tue Dec 20, 2011 9:32 pm

Re: Can I do a text search for "Is Not Blank"

Postby thomaskyhn » Sun Jan 19, 2020 12:46 am

SFA wrote:
thomaskyhn wrote:
SFA wrote:
Regex, for those that know it and understand the subtly different implementations that are in use, might be nice but I think that potential users, in Capture One world, might be a very small minority and usage rather rare.

Regular expressions are used in many kinds applications; I shouldn't think that Capture One users would be less familiar with them than other software users.


It sounds like you are quite familiar with REGEX Thomas.

I do not know with any degree of certainty but think it is probable that relatively few C1 users are familiar with REGEX - perhaps especially those who benefit from hand-holding quite frequently.

I work with a business analytical system that introduced REGEX about 4 maybe 5 years ago. Despite some quite comprehensive existing facilities some influential people at one or two major clients persuaded then to do the work and include REGEX abilities

Judging by observations in their forum and comments from a few staffers I know there were a few experienced users of the application who decided they ought to find out what REGEX was and a couple of younger members of the forum who had previous exposure to the use of REGEX and that was about it.

An experienced colleague of mine who at the time was responsible for about 100 users of the application working in a large organisation full of very smart people argued against its inclusion offering the right sort of benefits to typical users on the basis that none of his users currently used REGEX in any form nor understood what it was. He suggested that, in its raw form, it would most likely cause more support issues (or just unrecognised errors) from users who might try to use it occasionally and would therefore not be beneficial. He felt there should be better option offered for a product that was not intended to be something that required an IT support operation of skill in order for it to be successfully deployed.

I think he had a point.

The front end of a search engine in a product aimed at the general public is not really the place to expect users to have familiarity with REGEX in order to be able to find a couple of photos.

For example for the original search requested in this thread the chances are that most people's use would involve words that, in English, would contain at least one of the letters a,e,i,o,u and possibly a space.

So a saved preset search that was defined as

Content Description > contains > a
Content Description > contains > e
Content Description > contains > i
Content Description > contains > o
Content Description > contains > u

and another entry for "space"

using a Match set to "ANY"

would be expected to find all images with text in the Content Description field.

I feel fairly certain that similar approaches could be taken for the other supported languages.

What are your thoughts?

Perhaps it would only be beneficial to a small part of users, I don't know. You don't need to have a complete grasp of it though – I, for one, don't – simple expressions can also be useful.
thomaskyhn
 
Posts: 166
Joined: Wed Oct 24, 2018 5:50 pm
Location: CPH, DK

Re: Can I do a text search for "Is Not Blank"

Postby OddS » Sun Jan 19, 2020 9:32 am

thomaskyhn wrote:I shouldn't think that Capture One users would be less familiar with them than other software users.


I, for one, have used regexps regularly since the seventies, and I know them quite well. Jeffrey Friedl authored the better book "Mastering Regular Expressions" in the mid nineties. He also deserves mentioning for his long time contributions of an unknown number of photographer's utilities, often as add-ons/plugins to Lightroom. He has a web site and blog...

I never saw the SQL standard(s) include regexps, but I think some relational database systems include ways to get regexp functionality. SQLite, the C1 database system, does not include regexps, but SQLite can register external (system) functions for use in queries. Similar to, but not the same as SQL's stored procedures. An external function could probably offer regular expressions with SQLite (I never tried, I do regexps in script code).

However; SQL has the notion of NULL meaning "empty" or "nothing". My guess is that search criteria "WHERE column IS NULL" and "WHERE column IS NOT NULL" are among the most frequently used expressions in the world of SQL. If C1 does not allow search criteria "IS NULL" and "IS NOT NULL", it is certainly not a limitation imposed by SQLite.
OddS
 
Posts: 292
Joined: Tue Mar 25, 2014 10:28 pm

Re: Can I do a text search for "Is Not Blank"

Postby thomaskyhn » Sun Jan 19, 2020 10:44 am

OddS wrote:
thomaskyhn wrote:I shouldn't think that Capture One users would be less familiar with them than other software users.


I, for one, have used regexps regularly since the seventies, and I know them quite well. Jeffrey Friedl authored the better book "Mastering Regular Expressions" in the mid nineties. He also deserves mentioning for his long time contributions of an unknown number of photographer's utilities, often as add-ons/plugins to Lightroom. He has a web site and blog...

I never saw the SQL standard(s) include regexps, but I think some relational database systems include ways to get regexp functionality. SQLite, the C1 database system, does not include regexps, but SQLite can register external (system) functions for use in queries. Similar to, but not the same as SQL's stored procedures. An external function could probably offer regular expressions with SQLite (I never tried, I do regexps in script code).

However; SQL has the notion of NULL meaning "empty" or "nothing". My guess is that search criteria "WHERE column IS NULL" and "WHERE column IS NOT NULL" are among the most frequently used expressions in the world of SQL. If C1 does not allow search criteria "IS NULL" and "IS NOT NULL", it is certainly not a limitation imposed by SQLite.


Thanks for the Jeffrey Friedl reference; I wasn't aware of him.
thomaskyhn
 
Posts: 166
Joined: Wed Oct 24, 2018 5:50 pm
Location: CPH, DK

Re: Can I do a text search for "Is Not Blank"

Postby SFA » Sun Jan 19, 2020 12:42 pm

OddS,

The business oriented software I use is based on SQL driven databases at desktop and server level. To introduce REGEX capabilities they seem to have bolted in a utility of some sort that provides the sort of functionality you mention. REGEX is not a wheel that anyone really wants to re-invent. I think they had enough of that when they decided to build in a data extraction function for the text contents of PDF files.

Something they thought would take a few days - a couple of weeks at the most - turned into a saga fit for a Norse legend.

However the two most evident basic functions of their application are ETL projects and multi-source data analysis and reporting. There is more but those are core.

The NULL concept is an absolute requirement where one cannot be sure that a controlled database (where a bad value is absolutely NOT an option especially for numeric field, dates and and data files that must contain a value to be used reliable - like table driven codes) and the data form some external source will always be well aligned and correctly populated.

For optional text based comments fields there requirement is not so strong. In fact one could argue that to enforce NULL or not NULL in a field that would be perfectly acceptable for typical use as a blank field could be wrong in any situation other than ETL type activities.

Then there is the question about how many of the application users are what we might refer to as IT professionals at some level.

The application I am referring to is very much focused on empowering non technical people sitting at admin desks working with business processes that they knew to become departmental analysts in their own right without being exposed to the tech language of an IT role.

This works well for anyone with at least a little inclination towards the idea. Complex comparisons and selections, as can be derived via REGEX, are possible using careful constructions of individual searches - just as in C1.

As the product grew into large organisations and developed an increasing server based user base run mainly by IT pros the large spends of those organisations and the influence of the primarily IT buyers they employed led to requests for more complex solutions (especially of the type that require IT Professionals to understand, create and deploy for others!) and rather than build them in house it was decided to implement the REGEX engine option and see how that worked out for the Office Desktop user as well as the IT Pro departments.

As an aside, while all of that was going on along with a stack of other developments in the business, the prices increased significantly and they converted to a subscription pricing model. That seemed to change their traditional customer mix and so in turn changed the type of use input in terms of suggestions for enhancements for the product.

But I digress.

The point is that I don't really think that a Photo editing program has (or should have) the sort of problems that the NULL concept sets out to assist with. (Data field population checks and preventing erroneous values in calculated fields, mainly)

It may be that these days most of the world's population knows about NULL values and what they are for but I doubt it.

As for people using RAW converters ... people with an IT background may be well represented in the C1 user base and so the understanding of NULL may be higher than in the general population but I doubt the developers could assume that was a strong enough case to NOT provide a non-REGEX approach.

If one considers adding the REGEX functionality in addition to the 'normal' functions then there are development and continuing support costs (and possibly license fees?) to consider for something that, in the context of the product, is of somewhat limited use and hardly critical to the product's function and revenue generation ability. One can make at least a reasonable case for the functionality in a modern ETL system.


NOTE: For anyone reading this who is not familiar with the acronym ETL it stands for Extract, Transform and Load. Basically a process that take data from one or more sources, modifies of combines that data and presents it for loading into another system. There are variation on that theme. ETR for example may be referred to for Extract, Transform and Report. Much the same thing, different destination and output format. (Also, since we are in a Photography centric forum, nothing at all to do with ETTR or ETTL in photographic parlance! ;) )
SFA
 
Posts: 7720
Joined: Tue Dec 20, 2011 9:32 pm

Previous

Return to Workflow and Common Photography Exploration



Who is online

Users browsing this forum: No registered users and 4 guests