/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.


New Thread
X
Max 20 files0 B total
[New Thread]

Page: Prev [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] Next | [Index] [Catalog] [Banners] [Logs]


https://youtube.com/watch?v=J5I00p3KqvE
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v551a/Hydrus.Network.551a.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v551a/Hydrus.Network.551a.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v551a/Hydrus.Network.551a.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v551a/Hydrus.Network.551a.-.Linux.-.Executable.tar.gz

I had a great week mostly fixing bugs.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

First off, thanks to a user, you can now turn off the thumbnail 'fade' transitions under options->thumbnails. This will just make any thumbnail appearance or change happen instantly, in one frame.

Secondly, users who are on PyQt (macOS, and some who run from source), had some bad crashing issues last week. I changed one tiny thing, and it turns out PyQt had a big problem with it--Mr Bones and 'open externally' panels and the ratings hover in the media viewer all displayed wrong and caused program instability. Thanks for your reports, sorry for the trouble, and should be all fixed now!

Other than that, I just fixed a bunch of different little things. Some transparent image handling, a weird timezone calculation, database counting issues, and a ton of file move/copy failure handling. Check the changelog for details.

next week

I'd like to do something new, so maybe 'system:has transparency'.



https://youtube.com/watch?v=Tf8GG3gwam8
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v548/Hydrus.Network.548.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v548/Hydrus.Network.548.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v548/Hydrus.Network.548.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v548/Hydrus.Network.548.-.Linux.-.Executable.tar.gz

I had a good week. Thumbnail shortcuts are now customisable.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

First off, I cleared out a ton of bad thumbnail keyboard shortcut code and moved it all to the newer system under file->shortcuts. There is a new shortcut set there, called 'thumbnails', that governs opening the media viewer, the archive/delete filter, selecting files, and moving the thumbnail focus. Everything is now customisable (although I'd stay away from the thumbnail focus stuff unless you really want to wade through a big list), and I have added some new commands for 'select inbox', 'select trash' and similar, for more file selection options. This work clears out some really ancient code, and it should be much easier to extend thumbnail shortcuts in future, including hooking mouse-clicks into the customisable system and adding ctrl+selection logic.

Thanks to a user, we now have renderable krita files! We've got it set like PSDs for now, where you'll get 'open externally' on the preview and the full image in the normal viewer. As always, if you have files that don't work or render crazy, please send them in!

An issue with file export drag-and-drops sometimes giving the wrong file when those files share the same export filenames should be fixed!

Just a small thing, but a couple of places that were unintentionally sorting service lists in a random way now sort them alphabetically, as indended. The F9 new page picker was one of these, so if you have multiple local file domains, you may need to learn some new muscle memory!

next week

With luck I'll have some file storage stuff done, and I'd also like to clear a 'thumbnail fill' rewrite I've been planning.
thumbnail of why.png
thumbnail of why.png
why png
(111.42 KB, 1680x606)
Has this been fixed yet? Last time I asked was over a year ago.
These don't show up in the dupe filter unless I manually disband the dupe group, but how did they obtain it in the first place?
Happens regularly but not that often to be a major nuisance.

I had a good week. I ended up working on technical improvements, fixing some issues with deleted file tracking and exposing multiple maintenance timings for customisation. There is also a variety of optimisation that should slightly de-lag very heavy clients.

The release should be as normal tomorrow.

 >>/1550/
I am sorry, I cannot remember this well. Can you say more what is not working right here? It looks like you have a couple of files set as duplicates, and they are showing up in the search result. Have you never set these as duplicates, and they seem to have spontaneously duped together?

Or do you use the duplicate filter, and have seen files like this, but always set 'and delete the other', and so it is odd that this has happened and they remain in your 'my files'? Is that why there is only one pair when you do this (normally expansive) search?

If it is strange that these files are not showing up in the duplicate filter--that is intentional. The duplicate filter only shows potential duplicate pairs. Once those potentials are processed in the filter, they become like this pair--set as duplicates.

As a side thing, I am planning out a thumbnail expansion to give all thumbnails quick knowledge to all of their current file relationships, which should make some of these menus a bit cleaner and more naturally human (e.g. saying 'these two are duplicates' explicitly, rather than having to infer it from everything else going on). Unfortunately, I have to fetch this data on a per-file basis from the database on every right-click, so I can't do all the calculations I'd like.

 >>/1551/
So I avoided using the dupe filter altogether during all this time to make sure it has nothing to do with it and while the frequency of these weird dupes reduced, they continue to appear. Either way, I never encounter them in the filter before they show up here and I always delete the dupes I find.
These weird dupes differ only in compression as they usually involve an image imported (from API) a long time ago and its copy retrieved recently, probably because the server's compression format being updated. They have multiple image urls, some are matching, others different between the two.

So basically these are images that got assigned the same dupe group without going through the filter where I would have deleted one. Instead I have to look for these stealth dupes in a special search query.

 >>/1558/
Thanks. That's very strange behaviour. As far as I know, the client will only ever apply a duplicate pair relationship if you work through the duplicate filter or apply it manually on a selection of thumbnails, either through that menu or with a shortcut. I have no auto-dupe tech yet.

As a side thing, there is some transitive behaviour, so if A is a dupe of B, and you then set B to be a dupe of C, then A will be duped to C also, but of course this needs the triggering B->C relationship set in the first place, which it sounds like you are not doing. Also, duplicate relationships will survive files being deleted and re-imported, if that could be happening here.

If you can figure out a way to reliably reproduce this situation, or you gather any other information that would help me track this down, please let me know.



https://youtube.com/watch?v=yj2g8l1JqWI
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v550/Hydrus.Network.550.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v550/Hydrus.Network.550.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v550/Hydrus.Network.550.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v550/Hydrus.Network.550.-.Linux.-.Executable.tar.gz

🎉 Merry 550 🎉

I had a good week. I fixed some important bugs and reworked 'scale to fill' thumbnails.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

I fixed an issue with the login system where, for certain scripts, a job that was supposed to be waiting on an ongoing login process could nonetheless start early. All jobs are more patient about waiting for their specific login process to completely finish, no exceptions.

I added a maintenance system to explicitly close and clear persistent network connections. I had underestimated how aggressive my underlying libraries were at closing these no-longer-used 'keep-alive' network connections, and a half-dead object was occupying your OS's network slots for longer than desired. Network connections now close promptly five minutes after the last use.

I fixed some thumbnail display bugs, and, particularly, 'scale to fill' thumbnails are much more sensible behind the scenes. Previously, the thumbnail on disk was of the cropped visible region (which was a little complicated to figure out and maintain); now, things are simple--it just stores a full-image thumbnail at the appropriate larger scale and then draws the cropped area of that on demand. There are several technical benefits to this, and it saves some CPU in the long run, fixes some bugs for things like blurhash which need the full image to work properly, and it means an external process like the Client API can now ask for a thumbnail and get the full image rather than an arbitrary crop.

If you are set to 'scale to fill', your thumbnails will regenerate as you browse. Fingers crossed, you will not notice any visual difference. Let me know if the regeneration is annoyingly laggy--I didn't want to schedule a ton of regen CPU for you all this update, but I can figure a 'catch up in the background' thing out if it is a problem.

next week

More small work and bug fixing, I hope. If I have time and energy, some file storage overhaul work.

I had a great week fixing a ton of bugs. As well as a variety of small stuff, there's also a fix for the Mr Bones and top-right media viewer hover crashes that PyQt users have seen this past week.

The release should be as normal tomorrow.

 >>/1555/
Try here, looks like one as of two weeks ago:
https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/tree/master/Downloaders




https://youtube.com/watch?v=VR9oZZcdYNU
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v549/Hydrus.Network.549.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v549/Hydrus.Network.549.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v549/Hydrus.Network.549.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v549/Hydrus.Network.549.-.Linux.-.Executable.tar.gz

I had a good week, improving performance and adding some advanced tools for future testing.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

I fixed some deleted file records in the database. When you update, it'll take a moment to recalculate some things.

The options->maintenance and processing page has a ton of new settings to tune how hard the various background maintenance systems work and rest. These settings are mostly only for debugging, but if you are an advanced user and feel brave, you can play with them. If you have the 'memory explosion on PTR processing' problem, please check the changelog for instructions.

I optimised the taglist sort and update code a bit. Very large pages should be a little less laggy.

If you run from source, the 'easy setup' scripts have a couple new choices for setting up Qt, including choosing to type a specific version in. If you had the issue where your menus were in the wrong position, there is now a new (t)est Qt version (6.6.0) to try out, too.

next week

I got caught up in this 'background' work this week. I'd like to do something more visible and fun next week, but we'll see!



https://youtube.com/watch?v=Cjkjba-qbhE
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v547/Hydrus.Network.547.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v547/Hydrus.Network.547.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v547/Hydrus.Network.547.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v547/Hydrus.Network.547.-.Linux.-.Executable.tar.gz

I had a good week mostly fixing bugs.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

Most importantly, I banged my head against a persistent knot of mpv crashiness this week, and I actually got somewhere. A variety of situations where mpv could fail to load a weird file, which would then lead to hydrus program instability, are now much much better, usually no crash worries at all. Also, there's a little more error handling, stuff like a placeholder image to show instead of the broken file, and an error popup to tell you what happened. I cannot promise that this improves mpv on any system that already had trouble just getting it working (e.g. macOS, some Linux), but I think that heart-skipping moment of 'will it crash?' lag when an mpv load fails should be solved. Let me know how you get on!

Thanks to a user, we have support for 'djvu' files, which are basically an open source pdf format.

If you have a client with hundreds of search pages open, please let me know if your session is any less laggy, particularly if you get bumps of lag every five minutes or so. I rewrote some ugly code and think I eliminated some background CPU overhead for your situation.

There's a handful of smaller fixes and tweaks too, nothing super important, but check the changelog if you are interested.

next week

I'm sorry to say my past couple of months have been pretty low productivity overall. Just life stuff cutting at my time and energy, mostly. I'm going to keep my head down and not change anything drastic, but it has been on my mind, and I regret not being able to push on larger projects. We'll see how the new year goes, I think. Anyway, I expect I'll clear out some more jobs like this next week. I've also been dabbling with the idea of a new 'hash sort' to preserve the sort of a 'system:hash' or downloader highlight in various situations, so I might explore that.



https://youtube.com/watch?v=aFx7woWkZbc
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v546/Hydrus.Network.546.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v546/Hydrus.Network.546.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v546/Hydrus.Network.546.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v546/Hydrus.Network.546.-.Linux.-.Executable.tar.gz

I had a simple week. I mostly cleaned code, so there isn't much to talk about except for some bug fixes to recent systems.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

misc

I accidentally screwed up PSD thumbnail colours at the last minute last week while fixing something else. This is fixed, and any borked PSD thumbnails should regenerate themselves soon.

The new file history chart's cancel button works better, and I fixed some of the counting logic in the archive/inbox lines when you have a search filtering the results. Let me know if you still see archive numbers that are way too high.

If you have run into the new 'blurhash' thumbnails and don't like them, hit the new checkbox in options->thumbnails to turn them off!

next week

I really got lost in code refactoring this week. It is always worthwhile work to do, reshaping old bad ideas and preparing for the future, but it is unexciting, and with a janked out codebase like mine it often feels like bailing out the ocean.

For next week, I'd like to get back to my file storage system improvements, ideally getting background migration ready to go.
I had a good week. I fixed a bunch of bugs and optmised some code. Some mpv-related crashes should be completely gone, and there's also support for 'djvu' files.

The release should be as normal tomorrow.



https://youtube.com/watch?v=OLJhFROQg5A
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v545/Hydrus.Network.545.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v545/Hydrus.Network.545.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v545/Hydrus.Network.545.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v545/Hydrus.Network.545.-.Linux.-.Executable.tar.gz

I had a good week. I did some small things, and a user contributed some cool things!

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

blurhash

tl;dr: we have some special blurry thumbnails now, but you won't see them much yet.

Thanks to a user who did a bunch of research and work to get this going, the client now generates the 'blurhash' of all files that have a thumbnail. This hash is essentially a micro-thumbnail, only about 34 bytes, that shows a very blurry coloured impression of the general shape of the image or video. They are usually used as placeholders when it may take time to load something--you have probably seen something similar on slower news websites.

You won't see these in hydrus itself much, since thumbnails load fast enough that we don't have to worry about placeholders. In the advanced ways of seeing files you don't actually own, however, I will now show you any file's known blurhash rather than the default 'hydrus' thumb. Same deal for damaged/missing files--if I can't fetch a thumb, I'll now fall back to the blurhash.

The more important thing is these hashes are now available on the Client API, under the normal 'file_metadata' call. If you are implementing a browser or similar and have access to a fast blurhash library, try them out! I've scheduled all users to generate the blurhashes for their existing files in the background, which will take a few weeks/months for users with hundreds of thousands of files (although, if you are working with this and want to hurry it along, remember the queue is manageable under database->file maintenance.

other highlights

The file history chart (help->view file history) now has a search panel, just like Mr Bones! You can search your import, archive, and delete history for creator x, filetype y, or any other query you can think of. Some of the math here got a bit weird, and I am sure I have missed out several 'cannot provide good numbers for this domain' situations, so let me know if you get any really stupid results or outright errors. There's more work to do here, like a button to hide the search panel, which I hope to push on in the near future.

Thanks to the same user above, we also have epub support! No 'num_words' yet, but it turns out epubs are just zip files with some html, so I think it'll be doable in future. Also, some rare incorrect jpeg rotations (for 'MPO' jpeg files) are fixed.

If you right-click on a selection of files, the 'open->similar files' menu now has a 'in a new duplicate filter page' command. This will initialise the filter page just looking at those files, hopefully making it simple to clear out the potential duplicates in any particular selection.

Unfortunately, I am retiring the Deviant Art artist search and login script. DA have been slowly killing their nice old API, and the artist search just went. Individual page URLs still seem to work, but I suspect they will be gone soon. Gallery-dl have a nice DA solution that works with the new API, so if you can't find the same content on a more open booru, that's my best recommendation for now.

next week

I want to take a very simple cleanup week. Nothing too exciting on the changelog front, but I'll refactor some of the worst code into something nicer to work with.
I had an ok week. I mostly cleaned code, so the changelog isn't too exciting, but there's fixes for bad PSD thumbnails and the new file history chart's inbox/archive lines.

The release should be as normal tomorrow.



https://youtube.com/watch?v=T9IjJIX50Ls
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v544/Hydrus.Network.544.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v544/Hydrus.Network.544.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v544/Hydrus.Network.544.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v544/Hydrus.Network.544.-.Linux.-.Executable.tar.gz

I had a good week. File storage locations can now have max size limits. There is a security fix in this release, and users who run from source will want to rebuild their venvs this week.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

max size

This is just for advanced users for now.

So, in migrate database, if you have multiple locations set for your files, you can now, for all but one, set a 'max size', limiting the total mass of files it will store. The idea here is that if you have a 2TB drive and a 500GB one, then rather than having to keep playing with the weights to ensure the 500GB doesn't get too full, you can now set the 500GB location with a 450GB limit and hydrus will allocate that space and no more.

There are a couple of limitations. It isn't perfectly precise, so if your space is tight, give it a little padding (although remember that drives should be at least 10% free space anyway!). Also, it won't yet enforce these rules outside of the 'move files now' button. If you set a tight limit and then import many files, it will blow over that limit (at least until you open up migrate database and hit 'move files now' again to rebalance).

The next step here is to implement automatic background file migration for always-on checking so we can have these rules apply without regular user intervention. My aim is to get rid of the lagtastic 'move files now' entirely so no file migration blocks other use. I hope to have this done in the next few weeks.

webp

A remote execution (very bad) vulnerability was recently discovered in the main webp library. You probably noticed your chrome/firefox updated this week, which was fixing it. Our 'Pillow' image-loading library uses libwebp, and they rolled out an update themselves this week. The builds today incorporate the fix, so if you use them, just update as normal and you are good. If you run from source, reinstall your venv.

However, if you are Windows 7 (or otherwise on Python <=3.7), I am afraid you cannot get the fix! As I understand, it just won't work for you. There is now an additional choice in the 'advanced' setup_venv script to choose the older Pillow version, so you can keep running, but I'm afraid with this and other things you lads are now solidly in the phase of limited security updates. I will keep supporting you as long as it is generally reasonable, but if you still want to use Win 7, I think the train is running out of track.

misc

When images fail to render, they now show something proper in the media viewer. (previously they were just blank forever and made some error popups).

After last week's updates, Mr Bones gets a readability/layout pass.

The client should stop mistaking various text files for mpegs in the pre-import phase!

I think the Docker client wouldn't boot last week, same for anyone else without the PSD-tools library. Sorry for the trouble, should be fixed now!

next week

I'd like to add a file search to the file history chart and get stuck into more file storage infrastructure updates.
 >>/1538/
Hi dev,
I have a question about importing.
So I haven't been able to figure out how to import files along with their tags from boorus, so for now I'm using Grabber for that, which doesn't seem to have the concept of sidecar files so it's limited to the charlimit of the filename, but at least I can get some tags with careful pruning of irrelevant ones.

The problem is, when I go to import, I don't see any option to import filename as tags based on a separator character (in this case, just a space).
I could swear this feature was in older versions of Hydrus and I've used it before but now I can't find it.
Has it moved or am I just missing something? Thanks.

I had a good week. There's support for epub files, some new micro-thumbnail tech called a blurhash, and the file history chart now has a file search box.

The release should be as normal tomorrow.

 >>/1539/
I don't specifically remember this, but maybe there's an old saved regex rule tucked away in the filename tagging dialog somewhere. Drag some files on the client, then, on the window that pops up, click 'add tags/urls with the import', then select the tag service tab up top and then click the 'advanced' tab in the top of the bottom half. The 'regexes' box is probably what you are looking for. There's some saved favourite regex patterns off the 'regex shortcuts' button--maybe you had something saved there?

If not, forgive me for dashing this out, but I think you'd want a couple rules like:

(?<=\s).+?(?=\s) (gets words between spaces)
(?<=D:\\hydrus_imports\\123456\\).+?(?=\s) (gets the first word after the import directory tree prefix)
(?<=\s).+?(?=\.jpg) (gets the last word before the file extension)

A bit of playing around will be needed depending on your exact filename format. BTW, ChatGPT is great at working out regex patterns for you, or explaining existing ones you aren't sure about, if you describe exactly what you want.

 >>/1540/
Thanks dev, I didn't realize ChatGPT actually knew Regex I just assumed it would do its usual trick of lying kek
As I remember it, there was no saved pattern, I just exported and added [all tags] or whatever to filename, maybe ticked some box or filled some text field to set the separator character, and then there was a similar tag recovery system on import using the same interface essentially. I may have hallucinated this or it may be long-orphaned, this was quite a while back I last tried it out, I think back then I had suggested a system to add a character to separate namespaces from their tags so the system would recognize them on reimport.

Anyways I'll be sure to give your regex a try.

Hey I reinstalled Hydrus on linux Mint, tried  to import a test webm video and it imported but no sound when I play it. But there is a sound when I select to open in another program. Would anyone know the cause of this?




https://youtube.com/watch?v=3zw3sRLVPN0
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v543/Hydrus.Network.543.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v543/Hydrus.Network.543.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v543/Hydrus.Network.543.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v543/Hydrus.Network.543.-.Linux.-.Executable.tar.gz

I had a good week. Mr Bones gets an update. If you run from source, you might like to rebuild your venv today to get some new parsing support.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

Mr Bones

So, the statistics under help->how boned am I? now have an attached file search. The numbers will change based on whatever you search, so you can now see the average size of your pngs, the archive/delete ratio for creator x, how many times you have viewed pictures of sonic the hedgehog, or any other search you can think up. The default search is what the dialog was fixed to before, 'system:everything' on 'all my files'.

This turned out to be surprisingly complicated, and to stop it spiralling in complexity, I culled some of the odder scenarios. If you do a multiple-file-domain search, you won't get 'deleted' or 'earliest import' numbers. And if you search 'deleted from x', you are going to get some jank logic. EDIT: I also disabled the potential dupes count right before release, it was performing awfully on my large IRL database. Keep it simple, and let me know what turns up. I love these sorts of weird statistics.

I'd like to do the same for the file history chart soon. Should be a bit easier!

misc

PSDs should render more quickly and with less memory!

PDFs no longer spam the log on thumbnail generation!

Bitmaps files (.bmp) are now fully supported. Since the program started, these have been converted to png on import, mostly because the bmp format personally annoys me. This was always in conflict with the tenet that hydrus not alter files in any way, and I think it was ultimately a mistake. Bitmaps now stay as-is, and we've happily untied a couple of awkward knots in the code as a result.

If you are a downloader maker, there's a new 'datestring to timestamp (easy)' String Converter step. It uses an excellent external library ('dateparser') that can parse pretty much any datestring you can think of, including stuff like '2 hours ago'. If you run from source, this is the thing you are rebuilding for today. I have left all the existing 'datestring to timestamp' conversion steps in place for now, as '(advanced)', but if this new step integrates well, I think it'll supercede them and I may ultimately convert them all over.

next week

I'm going to have a busy week IRL, so work might be limited. I trudged through more file storage overhaul this week, and while I didn't finish the 'max num bytes' option as I had hoped, it is all ready behind the scenes. I just need to put some UI together and I think we'll have it working.
I had a good week. I fixed some bugs, improved some quality of life, and added a 'max size' option to locations in the 'migrate database' dialog.

The release should be as normal tomorrow.



https://youtube.com/watch?v=UZr6b4mBoxI
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v406/Hydrus.Network.406.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v406/Hydrus.Network.406.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v406/Hydrus.Network.406.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v406/Hydrus.Network.406.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v406.tar.gz

I had a good week. It is now easy to deduplicate queries in subscriptions.

subscriptions

Managing multiple subscriptions can get complicated. Figuring out which queries are in which sub, whether and where you have duplicates, is tedious. A recent bug with query pasting also could have introduced some duplicates within the same sub. This week should make it simple to manage.

The main manage subscriptions dialog now has a 'deduplicate' button. It lights up any time your subs of a particular downloader have the same query, whether that includes dupes within the same sub or across different subs. Click it, and it launches a thorough process to determine what you want to do:

- First, it asks if you want to deduplicate by case or not--e.g. whether "Samus_Aran" and "samus_aran" should be considered dupes, or only exact matches.
- Then it summarises which downloaders appear to have dupes, letting you choose which to work on.
- Then you have the option of selecting which of the queries you wish to deduplicate--e.g. if you want to do two runs to dedupe, say, artists to one sub and characters to another.
- Then you select which subscription(s) will retain the deduplicated queries.

Every step explains itself. Mostly it is simple, but there are some odd situations. When a sub dedupes queries within itself, it keeps the one(s) with the most files. I hope I have accounted for everything reasonable.

Adding to caseless matching, also added is a 'lowercase' button, which will coerce all the queries in the selected subscriptions to lowercase, just to help keep things neat. And lastly, when you paste queries into a sub, the 'already in the sub' test is now caseless.
full list

- subscription management:
- the manage subscriptions dialog now has a 'deduplicate' button. it is enabled whenever your subs of a particular downloader contain duplicate queries. it launches a semi-bananas but thorough 4-step process to ask if you want to do upper/lower-case deduplication, then which downloader, then which queries, then which master sub(s) to retain the queries
- subscription dedupe within the same sub keeps the query with the most files
- the manage subscriptions dialog also now has a 'lowercase' button that coerces all queries of the selected subs to lowercase
- when pasting a list of queries into a subscription, the 'already in sub' test is now caseless. pasting "Samus_Aran" into a sub already with "samus_aran" will not add anything
- .
- misc:
- url classes now have a checkbox to keep fragment data (e.g. "#kwGFb3xhA3k8B") during URL normalisation. this data is not sent to the server and is not useful for almost all sites, but for sites like Mega, it contains useful clientside javascript navigation or access info if you open the URL in your browser
- fixed video resolution parsing for some unusual SAR files. this stretches a video slightly, usually when the vid was created or converted with older analog tech (e.g. NTSC)
- fixed rating system predicate label for 'rated/not rated'
- the issue where miscounts in pending upload data would persist, sometimes leading to an annoying 'pending (13)' style menu that would not clear without debug action, is now fixed in a cheap way. on any upload action, this cached count is reset. a fix for the actual unusual miscount situation will have to come later
- the different in-memory manager objects now save changes at different time intervals--lightweight things like favourite searches still save not long after any change, but column widths, network sessions, and bandwidth use now save only every ten minutes
- I _may_ have fixed an issue with favourite tags not sticking correctly or resetting when added en masse via the tag right-click menu
- I believe I fixed a rare but permanent ui hang on highlighting a gallery or watcher when that same downloader was spamming through a largely 'already in db/previously deleted' list
- copying tags 'with counts' now works correctly for simple tag views (previously, it only worked for 'predicate' views)
- copying tags now preserves the tag order as in the list (previously, it did a human sort)
- to stop status-sorted gallery and watcher list entries bouncing around so much, they now just say 'working' in their status column when they are working. the highlight panel still reports granular file/gallery info. galleries also now say a more solid DONE when complete, to spot them more easily
- the gallery and watcher search/checking column now includes stop status in sort
- fixed the dowloader link in the help to https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/tree/master/Downloaders
- added that same link to the Lain dowloader import panel's help button
- updated cloudscraper to 1.2.46
- updated cloudscraper interfacing code to adapt for new reCaptcha->Captcha object names
- .
- boring code cleanup:
- refactored downloader gui code to its own file
- refactored network gui code to its own file
- refactored service gui code to its own file
- finished import reordering. now all files import in a cleaner order
- further reworked all hydrus imports to be more breadth-first, loading core modules earlier and catching potential errors in nicer places
- checkbox selection is now wrapped in the 'quick' dialog system, and all checkbox selections now use this single method
- simplified and unified a variety of layout code, and fixing some odd layout expanding bugs
- misc code cleanup
- deleted some old unused ui code

next week

Just doing small jobs. I'd like to catch up on the github issue tracker, and I'll maybe do some sibling prep work for a bigger job in v408.

I had an ok week. Some of the work I did (siblings improvements) is not ready to be turned on yet, so tomorrow's release mostly just has bug fixes, including several related to the media viewer's hover windows positioning and behaviour.

The release should be as normal tomorrow.




Post(s) action:


Moderation Help
Scope:
Duration: Days

Ban Type:


0 replies | 0 file
New Thread
Max 20 files0 B total
Refresh