/hydrus/ - Hydrus Network

https://youtube.com/watch?v=MgNCjYXCxOc
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v525a/Hydrus.Network.525a.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v525a/Hydrus.Network.525a.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v525a/Hydrus.Network.525a.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v525a/Hydrus.Network.525a.-.Linux.-.Executable.tar.gz

There was a problem with the initial v525, so I rolled back the OpenCV update in this new v525a. If you installed v525 and cannot boot, please delete 'install_dir\cv2\cv2.cp39-win_amd64.pyd' or just do a clean install! If you are an advanced Linux user and use the built release above, please check the note at the top of the old v525 release here: https://github.com/hydrusnetwork/hydrus/releases/tag/v525 . Thank you!

I had an ok week. There's a mix of small work and some library updates for everyone.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

new library versions

Some advanced users have been testing new library versions for Qt (user interface), OpenCV (image processing), and, on Windows, mpv (audio/video display). There haven't been any reported problems so far, so I am rolling these new versions into the regular build and automatic setup scripts for source users. If you use one of the builds above, you don't have to do anything special to update--with luck, you'll simply get some subtly improved UI, images, and video. If you are a source user, you might like to run the 'setup_venv' script again this week after pulling, and it'll update you automatically. If you are a Windows source user, you might also like to get the new mpv dll, which requires a simple rename to work, as here: https://hydrusnetwork.github.io/hydrus/running_from_source.html#built_programs

I don't expect this to happen, but if you are on an older OS, there is a chance you will not be able to boot this release. If this is you, make sure you make a backup before updating, and then if you have problems, roll back and let me know, and we'll figure out a solution.

If you can't boot after an update, but a fresh extract on your desktop works fine, then you probably have a dll conflict, so run a clean install: https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#clean_installs

other highlights

I fixed some more time/calendar stuff. Some longer time durations now compensate for leap years and span month-deltas more precisely.

In the media viewer, clicking the 'exif and other embedded metadata' button up top on jpegs now shows their subsampling (e.g. 4:4:4) and if they are progressive.

I fixed an awful bug related to deleting the new inc/dec rating services. If you deleted one of these and found you couldn't load many files, sorry for the trouble--the underlying problem is fixed, and updating will correct any bad records!

The Sankaku downloader gets some new download objects this week to handle their recent shift to md5-based Post URLs. Unfortunately, this is going to cause any sank subscriptions to go slightly bananas on their next check, hitting their file limits since they think they are seeing new URLs. There's no nice solution to this, so you will pretty much just want to wait them out.

next week

More small stuff like this I think. Maybe I'll squeeze some API work in.

https://youtube.com/watch?v=UdiwBiw5dyo
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v524/Hydrus.Network.524.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v524/Hydrus.Network.524.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v524/Hydrus.Network.524.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v524/Hydrus.Network.524.-.Linux.-.Executable.tar.gz

I had a good week working on bug fixes, quality of life, and more timestamps.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

timestamps

I've added timestamps to the sidecars system. You can import/export any timestamp the file has, so it is now possible to migrate various times (particularly archive and imported) from one client to another. I still need to add some more things to sidecars, e.g. inbox status and ratings, but we are almost at the point where we can do complete client-to-client transfers of all the major metadata. I have plans to make this ultimate goal simple with templated one-click jobs and no need to repeat the file import--just a raw JSON file metadata import/export.

Also, in the duplicate system, file modified dates can now be merged, and by default they will--for 'this file is better', then an earlier timestamp on the worse file will be applied to the better file, and for 'they are the same', both files will get the earlier of the two. If you don't want this to happen, check your duplicate merge options and set 'make no changes'.

Also in duplicates, if you have URLs set to merge, then any associated domain timestamps with those URLs will be earliest-merged too.

Lastly, the program can broadly support pre-1970 timestamps now. 1970-01-01 is a special date in computing, and supporting times before it can be tricky, but I've fudged some general support. It is a bit of a meme, but maybe it'll be neat to apply '1503' to old paintings. The code I was working with says it is good back to 1 AD, but it seems like it can actually go further, ha ha. Anyway, have fun, and let me know if it fails. I know that the '1,504 years 2 months ago' strings are a little inaccurate--it turns out I haven't been counting leap-year days in those calculations, and it shows on the longer durations.

other highlights

Export Folders now have popups while they work. You can turn them off for regular runs, but now Import and Export Folders force-show their working popups when you manually start them from the file menu. These popups have stop buttons, so you can cancel Export Folders mid-job now too. Also, I removed the stupid and confusing 'paused' vs 'run regularly?' duality. Export Folders now just have 'run regularly?'.

I added some special predicate labels, 'system:ratio is square/portrait/landscape', for =/taller/wider than 1:1. They are in the quick-select list for system:dimensions too.

next week

I pushed it too hard this week and exhausted myself. Thankfully, next week is due to be code cleanup, so I'll try and take it easy and just dive into some simple refactoring work and do some small jobs here and there.

https://youtube.com/watch?v=LGs_vGt0MY8
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v523/Hydrus.Network.523.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v523/Hydrus.Network.523.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v523/Hydrus.Network.523.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v523/Hydrus.Network.523.-.Linux.-.Executable.tar.gz

I had a great couple of weeks working on file timestamps.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

timestamps

Unfortunately, although I did a ton of work here, not much of it is interesting or even visible! Essentially, two weeks ago, the different timestamps your files have--stuff like import time and archived time--were stored all over the place, and now they are stored together. When they are consulted, it now all happens down the same pathway, and I have written a proper update pipeline using this new system.

The user upshot is that you can now right-click->manage->times on a file and edit all its timestamps. This means archived, imported, deleted, previously imported (for undelete), file modified, domain modified, and last viewed times. This new dialog is basic and only for single files for now, but we can now see these things any time and edit little problems. If you end up using this dialog a lot, there's a media shortcut action, 'manage file times' for it.

I've also added 'system:archived' for the 'system:time' stub. It works just like the other time predicates.

I had hoped to integrate this new timestamp manager and update pipeline into the sidecar and Client API systems, but there was so much cleanup to do that I just ran out of time.  I'm very glad I did this work, and there's now a much nicer base to work with and extend time tracking in future--like if I want to figure out a 'fill in some good guesses for retroactive archived times', it wouldn't be so much of a hack now--but there is more to do.

other highlights

All multi-column lists across the program now show ▲ or ▼ on the column header they are currently sorted by. This is something I meant to add for ages; now it is done.

All menus across the program now send their commands' longer description text to the main window's status bar. They also show it on tooltips now. These descriptions have been in the program the whole time, but many were difficult or impossible to see. Let's see how annoying the tooltips are--I'll add an option or turn them off if they keep getting in the way.

I fixed a stupid bug from v521 that was causing the advanced deletion dialog to always set the default reason. Sorry for the trouble--it was a testing issue!

next week

A mix of small work, just general catchup. I'd like to also get timestamps working in sidecars and/or the Client API, but we'll see.

Please feel free to ask questions about hydrus here.

As a reminder, the help and getting started guide is here:

https://hydrusnetwork.github.io/hydrus/help/index.html

https://youtube.com/watch?v=2AjrLci_AL8
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v522/Hydrus.Network.522.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v522/Hydrus.Network.522.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v522/Hydrus.Network.522.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v522/Hydrus.Network.522.-.Linux.-.Executable.tar.gz

I had a good week mostly working on technical things for advanced users.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

notes in sidecars

You can now import and export file notes with sidecars! There's a technical compromise I had to make here, which is that a note in a sidecar comes in the form 'name: text', both for import and export. If you want to import a whole bunch of notes, you'll need to wangle them into this format, or, if you can, use string processors to get them into that format. There's an escape sequence, ':\ ', for if your name has a colon in, too. Have a play with it and you'll see how it works.

The changelog and a new section in the sidecars help talks about it more, including issues about multi-paragraph notes. I suspect I'll be revisiting this, so let me know how it works for you!

neat user submission

This is advanced.

A user contributed two cool new things for the parsing system:

First, there's a new content parser type, 'http headers', which lets you parse something to be submitted in all subsequent URLs created by the current job. It should let us figure out some unusual tokens and pseudo-login issues.

Second, there's a new String Conversion type that lets you calculate the hex hash of any text for the normal hydrus suite of hashes--md5, sha1, sha256, sha512.

more http header stuff

This is advanced.

On my end, I added full custom http header management to the Client API this week. Basically anything global/domain related that you see under network->data->review http headers can be done via API now. This has been a long time coming, and I am glad it is finally out the door. It should allow for setting up some complex site access via the API.

next week

Next week is a 'medium size job' week. I'm not sure what I want to do. I've been thinking of a complete pass to let us edit timestamps, but there's a million other things in my list, so I'll think about it. I'm still a little busy with some IRL stuff, so it might end up being a two-week release.

https://youtube.com/watch?v=F7gXLqMe5Bw
Sorry these links were for v520 by accident earlier!
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v521/Hydrus.Network.521.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v521/Hydrus.Network.521.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v521/Hydrus.Network.521.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v521/Hydrus.Network.521.-.Linux.-.Executable.tar.gz

I had a good week mostly cleaning things and fixing some unusual issues.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

Building on last week's custom 'sibling' connection, options->tag presentation now lets you edit the 'OR' connector too, and you can now set a specific colour for them both using the existing namespace colours.

If you don't like file viewing statistics tracking your viewing in the archive/delete filter, you can now turn it off. Also, when you are looking at media with duration, the 'max' time it can record for that video or audio is now the larger of your 'max' setting and five times' that duration. So, if you are capped to ten minutes max normally, but you loop or skip around on a twenty minute vid for thirty mins, it'll save the full thirty mins. I'd like to add some pause tech in future too, to account for minimising and pausing media with duration. Thanks to the user who provided feedback on this stuff.

The file delete dialog should be better, in simple and advanced mode, at not overwriting existing delete reasons, which happens most when you manually delete from the trash. If you use this stuff a lot, let me know how it goes.

The rest is a bit advanced and niche. Hydrus parsers can now support unnamespaced tags with colons in, essentially an extension of the existing ':p' tags, and users can enter these tags too by starting a tag with a colon. If you make parsers, check out the changelog for how tag content parsers can now support 'any' vs 'unamespaced' namespace. I also fixed some weird file domain handling in importers.

next week

I really didn't have time for everything I wanted this week, so I'd like to figure out custom/remembered 'reasons' for the petition workflows, more file viewing stuff, http header editing in the API, and maybe notes support in the sidecar system.

https://youtube.com/watch?v=outcGtbnMuQ
If you are an engineer but didn't see the above vid yesterday, I strongly recommend it, the whole thing. We're clearly in the inflection point of this tech right now, and if you aren't experimenting with integrating some variant of it in some way into your work workflow, I think it is time. I just started using it for some scripting jobs I had been putting off this week and it honestly really helped out. There's problems aplenty in the whole space, but I'm pretty AI-pilled right now.
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v520/Hydrus.Network.520.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v520/Hydrus.Network.520.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v520/Hydrus.Network.520.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v520/Hydrus.Network.520.-.Linux.-.Executable.tar.gz

I had a good week. There's a mix of bug fixes and improvements to quality of life, mostly in the tag autocomplete interface.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

So autocomplete has a couple new things. First off, there's a new 'all files ever imported/deleted' entry under the file domain button. This searches everything your client has ever seen, runs super fast, and works with 'all known tags'. Also, when editing tags, I've tightened up the selection rules and reduced the sibling spam--now, you should get the 'ideal' of what you typed at the top, then what you actually typed (if that differs), and then normal results. Also, the various paste buttons tucked into edit autocompletes now only ever add items--if any of what you paste is already in the accompanying list, it won't be removed now.

And I changed the '(displays as xxx)' sibling indicator. It is now a simple unicode arrow, and because I wanted to play with gradients, I made it fade its colour from one namespace to another. If you don't like the text or the fade, you can change them under options->tag presentation.

There's a new Deviant Art downloader, thanks to a user's work, that pulls the 'original' quality from the 'download' button, which some creators turn on for logged-in users. There's also five new File URL URL Classes, which you can search with system:known url, that will categorise your existing DA downloads according to their quality. Enterprising advanced users might like to play around with this!

next week

Next week is slated for code cleanup. I want to do some boring database breakup work!

https://youtube.com/watch?v=NNNRlPfoC7I
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v519/Hydrus.Network.519.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v519/Hydrus.Network.519.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v519/Hydrus.Network.519.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v519/Hydrus.Network.519.-.Linux.-.Executable.tar.gz

I had a good couple of weeks prototyping a new rating service.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

inc/dec ratings

So, under services->manage services, there's a new 'inc/dec rating' (for increment/decrement). It works pretty much like all the other ratings, and goes in the same places, but it is a simple number counter--left-click to add, right to subtract. Middle-click lets you edit directly. I've wired it up for system:rating, too, with only one oddness--since every file starts at count 0, there's no concept of 'not rated' for this service.

If you want to count how often an image makes you smile or anything else, give it a go! I mostly did this for fun, but let me know how it works out.

other highlights

The manage siblings and parents dialogs now both try to show more of the full 'chain' of related pairs as you enter new tags. The manage parents one also has a new checkbox that let's you decide if you want to really show everything, including cousins (which can be overwhelming on the bigger tags), or just (grand)children and (grand)parents. I'm planning an overhaul to these dialogs, one where they don't have to load everything on boot, and this is a first step.

I fixed my native animation renderer drawing things too small when your UI scale is >100%!

Server janitors get a nicer account modification dialog this week, and a return of the old 'superban' functionality, which deletes everything an account has uploaded. Check the changelog for full details!

next week

I'm sorry to say I'm exhausted and lagging a bit. I'll clear some small work, nothing too special!

https://youtube.com/watch?v=lDWfgUjlEMk
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v518/Hydrus.Network.518.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v518/Hydrus.Network.518.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v518/Hydrus.Network.518.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v518/Hydrus.Network.518.-.Linux.-.Executable.tar.gz

I had an ok week. Tag autocomplete search should be a bit nicer to work with today!

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

tag autocomplete

I rewrote the part of tag autocomplete search that handles showing you early results. The whole thing works better and is smoother.

First off, every time you type something like 'cat' and the search is taking a while, you should now always get 'cat' and 'species:cat' and anything else that might match, with counts, and your selection will be preserved when the full results come in. Quick-entering a tag on both search pages and the manage tags dialog should be faster, easier, and more reliable. This is roughly how things used to work, a long time ago, but the various updates and new options I've added over the years made it patchy and buggy--it should be working again, and with some new quality of life improvements.

Secondly, I've improved sibling lookup in the manage tags dialog. You should now always see siblings, on the pre-results above and the final results, even if the siblings have no count. I think we might want some options for handling how these display and rank (e.g. 'always put the ideal of what I typed at the top of the list'), so see what you think.

Lastly, when you search for tags on a page of thumbnail results, you now get some pre-fetch results very quickly! The full results, which have extra sibling lookup data, will fill in a few seconds later, but, fingers crossed, there will be no more huge waits for anything on thumbnail tag lookups!

misc

The system:time predicates now support hh:mm, like '10:30pm', on the calendar search! Should be easy to cut a day in half now.

I believe I have optimised some content update code. If you have a client with a large page of thumbnails, particularly if that page uses collect-by, let me know if your subscriptions and/or 'upload pending mappings' run a bit smoother.

If you are an advanced Windows user, please check out the bottom of the changelog for an mpv test. I'd like to test out a new dll on a wide range of machines.

next week

My ongoing family IRL stuff is winding down. I'm still going to have extra responsibilities for another month or so, but it shouldn't be too bad. I'd like to have a medium-size job week, and I want to have some fun, so I'm going to try to make a 'click to increment' integer rating service.

https://youtube.com/watch?v=q4veLQfXFLI
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v517/Hydrus.Network.517.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v517/Hydrus.Network.517.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v517/Hydrus.Network.517.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v517/Hydrus.Network.517.-.Linux.-.Executable.tar.gz

I had another short week, but I'm generally happy with the work, which was a mix of different things.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

Thanks to a user's work, Export Folders now support exporting to symlinks! If you are on Windows, you may need to run hydrus in Admin mode for any symlink exporting to work (or, on Windows Pro, I believe you can mess around with Group Policy Editor to give normal user accounts the permission).

The newly upgraded 'Related Tags' shouldn't suggest superfluous siblings or parents any more! I'm pretty sure I got the logic right here, but let me know if it is still suggesting the wrong things somewhere.

Hydrus can now show a checkerboard pattern behind a transparent image instead of the normal background colour. There's two new checkboxes, in options->media and ->duplicates, to control it--by default I have set it just to show in the duplicate filter, where it is useful to have transparency pop when you are comparing two similar files.

next week

Yet again, I'm a little busy with IRL again next week, but otherwise should be normal. I'll do more work like this! Thanks for your patience!