icc profile and pixel hash
This is mostly database prep for future duplicate system expansions.
The client database now records whether still images have an ICC profile, and it also saves data for 'these images are exact pixel duplicates' decisions. On update, all your existing files will be queued for scans to fill in this data in the background. Anything with an ICC profile will also regenerate its thumbnail. You don't have to do anything, this will all happen automatically over the coming week(s).
In time, you'll be able to search for images with ICC profiles with the new 'system:has icc profile' search predicate. This predicate is weird and advanced, so I think I'll hide it away soon under an umbrella for advanced stuff.
The 'exact pixel duplicate' data will be useful in the near future, when I expand the duplicate system to find (and optionally automatically merge) certain pairs that are perfect visual dupes.
full list
- misc:
- fixed a recent bug in wildcard search where 't*g' unnamespaced wildcards were not returning namespace results
- sped up multi-predicate wildcard searches in some unusual file domains
- the file maintenance manager is now more polite about how it works. no matter its speed settings, it now pauses regularly to check on and wait until the main UI is feeling good to start new work. this should relieve heavier clients on older machines who will get a bunch of new work to do this week
- 'all local files' in _review services_ now states how many files are awaiting physical deletion in the new deferred delete queue. this live updates when the values change but should be 0 pretty much all the time
- 'all known files' in _review services_ also gets a second safety yes/no dialog on its clear deleted files record button
- updated the gelbooru 0.2.x gallery page parser, which at some point had been pulling 'delete from favourites' links when running the login-required 'gelbooru favorites by user id' downloader!!! it was deleting favourites, which I presume and hope was a recent change in gelbooru's html. in any case, the parser now skips over any deletion url (issue #1023)
- fixed a bad index to label conversion in a common database progress method. it was commonly saying 22/21 progress instead of 21/21
- fixed an error when manage tags dialog posts tags from the autocomplete during dialog shutdown
- fixed a layout issue with the new presentation import options where the dropdowns could grow a little tall and make a sub-panel scrollbar
- added handling for an error raised on loading an image with what seems to be a borked ICC profile
- increased the default per-db-file cache size from 200MB up to 256MB
- .
- some new options:
- the default tag service in the manage tags dialog (and some similar 'tag services in a row of tabs' dialogs) is reset this week. from now on, the last used service is remembered for the next time the dialog is opened. let's see how that works out. if you don't like it, you can go back to the old fixed default setting under the 'tags' options panel
- added a checkbox to the 'search' options panel that controls whether new search pages are in 'searching immediately' or 'searching paused' state (issue #761)
- moved default tag sort from 'tags' options panel to 'sort/collect'
- .
- deleted files and ipfs searchability:
- wrote a new virtual file service to hold all previously deleted files of all real file services. this provides a mapping cache and tag lookup cache allowing for fast search of any deleted file domain in the future
- ipfs services also now have mapping caches and tag search caches
- ipfs services are now searchable in the main file search view! just select them from the autocomplete dropdown file domain button. they have tag counts and everything
- it will take some time to populate the new ipfs and deleted files caches. if you don't have much deleted files history and no ipfs, it will be a few seconds. if you have a massive client with many deleted/ipfs files and many tags, it could be twenty minutes or more