/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.


New Thread
X
Max 20 files0 B total
[New Thread]

Page: Prev [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] Next | [Index] [Catalog] [Banners] [Logs]


https://youtube.com/watch?v=q4veLQfXFLI
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v517/Hydrus.Network.517.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v517/Hydrus.Network.517.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v517/Hydrus.Network.517.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v517/Hydrus.Network.517.-.Linux.-.Executable.tar.gz

I had another short week, but I'm generally happy with the work, which was a mix of different things.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

Thanks to a user's work, Export Folders now support exporting to symlinks! If you are on Windows, you may need to run hydrus in Admin mode for any symlink exporting to work (or, on Windows Pro, I believe you can mess around with Group Policy Editor to give normal user accounts the permission).

The newly upgraded 'Related Tags' shouldn't suggest superfluous siblings or parents any more! I'm pretty sure I got the logic right here, but let me know if it is still suggesting the wrong things somewhere.

Hydrus can now show a checkerboard pattern behind a transparent image instead of the normal background colour. There's two new checkboxes, in options->media and ->duplicates, to control it--by default I have set it just to show in the duplicate filter, where it is useful to have transparency pop when you are comparing two similar files.

next week

Yet again, I'm a little busy with IRL again next week, but otherwise should be normal. I'll do more work like this! Thanks for your patience!
I had an ok week. There's a mix of small fixes, the addition of hours:minutes to system:time calendar predicates, and a neat overhaul of how autocomplete tag search presents its pre-fetch results on slow searches.

The release should be as normal tomorrow.



https://youtube.com/watch?v=d1zDNBIswek
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v516/Hydrus.Network.516.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v516/Hydrus.Network.516.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v516/Hydrus.Network.516.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v516/Hydrus.Network.516.-.Linux.-.Executable.tar.gz

I had a good week that was unfortunately short again. There's some options to tune the new 'related tags' search and some misc little work.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

related tags

I have added three new settings to options->tag suggestions for related tags. They are fairly advanced and deal with the technical details of the search, so I only recommend poking around if you have been enjoying this system and want to change how it works.

First off is a 'concurrence threshold'. This is a percentage throttle that determines how 'related' a suggestion tag needs to be to its search tag to be included. The higher you make it, the fewer--but more relevant--suggestion tags you will get.

Next are two controls that boost or reduce the power of different namespaces in the search. If you want the character tags already on the file to do most of the searching, or if you want to see creator tags most of all, you can alter the weights here. You can also easily set a zero weight to say 'I don't want to see any filename: tags suggested' and so on. I have set them up with some reasonable defaults, so tweak what I've put in and see what you like.

I also fixed a stupid bug where you couldn't hide related tags while in side-by-side mode. Sorry for the trouble!

misc

APNGs that play only once now do that!

The new sidecar system now has import/export/duplicate buttons, so you can save and transfer your work! A favourites system will come in the future too, I'm sure.

The Client API has some more 'king' info in the 'get_file_relationships' command.

next week

The big IRL thing that I had to do is now behind me. I've got about six more weeks of lighter 'hydev has to do some family responsibility stuff', and then I am back to normal schedule. Next week I would like to continue on the small jobs, just to catch up on things that I have recently missed.
I had another light week. There's some misc fixes, symlinks for export folders (thanks to a user!), a new way to show image transparency (which will particularly help in the duplicate filter), and I finally removed superfluous sibling and parent suggestions from 'related tags'.

The release should be as normal tomorrow.



Tired of spending hours searching for voiceover artists? Save time and money with Speechelo. Instantly transform any text into a human-sounding voiceover with just 3 clicks.

Speechelo uses AI to generate professional-quality voiceovers in various languages and accents. Choose from male or female voices, and get your voiceovers done in minutes, not hours or days.

Transform your text into a professional voiceover now with Speechelo.

https://3e9bfyidnydw3k8l34n2v1snci.hop.clickbank.net

Best regards
Thor Andersson


https://youtube.com/watch?v=Q4brYfcHQR0
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v515/Hydrus.Network.515.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v515/Hydrus.Network.515.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v515/Hydrus.Network.515.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v515/Hydrus.Network.515.-.Linux.-.Executable.tar.gz

I had a challenging week with little work time, but I polished last week's related tags work.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

related tags

I've improved the 'related tags' test I did last week and brushed up the UI, and I'm happy enough with it that I am making it default-on for all users. You will see it after you update in your manage tags dialogs. If you do not like it, you can turn it off or customise its timings under options->tag suggestions!

For the technical details, I've cleaned up the new algorithm a bit, made it run faster in all cases and massively faster for very large-count tags, and now it works on the same 'cancel after 1,200ms' tech the old buttons ran on. Those buttons now use the new algorithm exclusively, there's a new label that reports how particular searches went. There's also a 'my files' vs 'all known files' on/off button to change the search domain ('new 1' vs 'new 2' in the test last week).

Also, if your suggested tags are set to show 'side-by-side', they now get nicer boxes around them, and if they are set to show in a notebook, you can set which is the default page under options->tag suggestions. Let me know how you get on with it!

next week

Unfortunately some stressful IRL landed on my head this week, and I'll have some extra responsibilities for the next 6-8 weeks. Nothing dreadful, but it will cut into my work time a bit. Please expect some more light and possibly skipped releases for the next bit, thanks!

Next week I'll see if I can just do some simple code cleaning and more small jobs.
I had another light week. I added some tuning options for the new 'related tags' search and cleared a handful of misc small work.

The release should be as normal tomorrow.



https://youtube.com/watch?v=PjfkAuUUq5w
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v514/Hydrus.Network.514.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v514/Hydrus.Network.514.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v514/Hydrus.Network.514.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v514/Hydrus.Network.514.-.Linux.-.Executable.tar.gz

I had a good couple of weeks doing a variety of work. The changelog is long, mostly fixes and improvements that don't change much, but if you use the twitter downloader or the Client API, make sure to read before you update.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

twitter

Twitter changed their policies regarding third-party interfaces recently, and it seems as part of that they took down the 'syndication' API our nice twitter downloaders were using. None of them work any more!

A user has kindly figured out a patch, which today's update will apply. It will throw up a yes/no dialog asking whether to do this, which almost everyone will want to click 'yes' on.

Unfortunately, this downloader can only get the first 0-20 tweets in the latest 20 tweets and retweets an account makes!! That's it! When you update, please check your twitter subs to make sure they moved to this new downloader correctly, and you might like to tell them to check more frequently.

I expect twitter to continue in this direction. I don't know if nitter or any other mirrors will be a viable alternative. If any of your twitter subs get posted in any normal booru, I strongly recommend you pursue those instead!

Client API

I am retiring some old data structures from the Client API today. If you use hyshare, please make sure you update to 0.11.2 or newer. If you use hyextract, please update to 0.4.1. If you use Hydrus Companion, I think you'll be good with the latest release for everything. The main changes on my end are to do with tag-viewing and tag-editing, so if you do a lot of that, make sure you have a backup and do a test of everything after you update.

If you have custom scripts, check the changelog. Mostly services are going to be referred to with service_keys in future, so I am removing 'service_name' structures and only supporting 'service_name' arguments in a legacy state. Please move to the /get_service(s) calls to figure out the service keys you need to use, as I will slowly retire the last legacy support later this year. Note that all users can now copy their service keys from review services.

As a side thing, I also rewrote how v413's /set_file_relationships call works. The old 'list of lists' parameter should still work, but a user reminded me that an Object with proper names would be a lot easier to extend in future, so I just rewrote it. This call is sufficiently new and advanced-user-only that I'm ok changing so quick, but if you set something up for it already, check the docs!
related tags

EDIT: In final testing, this stuff was working slow IRL for me, so I reduced its search space for today. You are going to see a lot of 'search domain is too large' on the PTR. I'll keep working!

Thanks to a user who sent in a great proof of concept, the 'related tags' column (which shows in manage tags and you can turn on under options->tag suggestions) has some new buttons to test. I called them 'new 1' and 'new 2'. The first works fast but may have limited results on repositories, the second--which only shows on repositories--can only search small-count tags but gives deep results.

Secondly, if you select some tags in the main tags list, the 'related tags' search will now only search those tags! If you can't remember a character name, then select a 'series' tag and click for a related tags search, and you'll get a bunch of them!

Thirdly, these new search buttons now work on manage tags launched from multiple thumbnails!

Please give this a go and let me know how you get on with IRL data. The quality of the new search results is better than my old routines, but I need to do a bunch more tuning and optimise their search. Are there too many recommendations (is the tolerance too high?)? Is the sorting good (is the stuff at the top relevant or often just noise?)? Which source tags (e.g. 'page:x') give the worst suggestions?

I'm sufficiently happy with the new performance, particularly with the tag selection, that once this is working nice, I think I'll make it default-on for everyone. Lots of people missed this system, and I always thought it was a bit jank.

other highlights

I wrote some sidecar import/export help here! https://hydrusnetwork.github.io/hydrus/advanced_sidecars.html

I optimised the how the thumbnail cache and downloader import queues do their normal work, particularly when they get tens of thousands of items. Big clients, which may have very busy versions of these objects, should be a little less laggy. Let me know if you notice any difference!

Downloaders that get from multiple sites in one query (aka NGUGs) now do their searches one site at a time, not interleaved (e.g. they now search sites A and B as AAABBBB, rather than ABABABB)!

next week

I've been semi-ill this past two weeks with some long-running something, and I pushed myself a bit hard getting all this ready in time. Next week will be some simple code cleanup, nothing too clever!

 >>/1453/
twitter gay, thanks for fix
I suppose this means my nitter subs are broken too?

On a different topic, does Hydrus do anything with DeepBooru or are there any extensions for it you know of that I can maybe use to try some autotagging?

 >>/1455/
Is DeepBooru the same as Deep Danbooru, or the same basic model, so you could switch it in place? Hydrus-dd offers several ways to plug into Deep Danbooru, here:

https://gitgud.io/koto/hydrus-dd

I was talking with someone else today about how I'd like to do more with this sort of tech in future, just making it easier to work with the sidecars you need or the Client API whatever. I'm sure image-to-text is going to benefit from all this text-to-image tech that's now floating around, especially now it is becoming reasonable to train your own models to do stuff. Hydrus as a big metadata-rich archive seems like it plugs into that tech pretty well!

I unfortunately had a light week. The new 'related tags' search is polished and turned on by default for all users, and I fixed some small bugs.

The release should be as normal tomorrow.



https://youtube.com/watch?v=D5T9RByz9i8
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v513/Hydrus.Network.513.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v513/Hydrus.Network.513.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v513/Hydrus.Network.513.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v513/Hydrus.Network.513.-.Linux.-.Executable.tar.gz

I had a good week focusing on adding the duplicates system to the Client API. There is nothing else, so today's release is only for advanced users.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

duplicates with Client API

The main duplicate commands are now available. Obviously check the changelog and Client API help out for full explanation, but the general gist is 1) There is a new 'manage file relationships' permission to gain access, and 2) You can fetch and create relationships, but you can't delete them yet. I will add these commands in the future.

https://hydrusnetwork.github.io/hydrus/developer_api.html#managing_file_relationships
https://hydrusnetwork.github.io/hydrus/duplicates.html#duplicates_advanced

There's some new enums to grapple with, and you can be dealing with two searches at once when fetching potential pairs. Please have a read, try things out, and let me know where it goes wrong. I expose the ugly technical side of things, so if it isn't clear what a king is, I may need to brush up the duplicates help etc...

I expect to keep working on this in the near future, including adding more search tech as I do related 'mulitple local file services' expansions to the Client API, and in the more distant future as I expand alternates. While I was in here this week, I realised I could brush up and clarify some of the edge-case logic too, as I wasn't sure myself on every situation. Again, let me know what seems confusing or wrong for you, and we'll iterate.

next week

A mix of small jobs, and it would be nice to add some remove/dissolve commands for this new duplicates Client API.

On a side note, I migrated to a new dev machine this week, upgrading from a Minis Forum X35G to a TH80. I'm pretty sure the old machine was a 2-core CPU while the new is 8, ha ha ha, so my working session feels super smooth now. I recommend these little machines if you need a decent office computer (the old one also drove two 4k@60Hz monitors no problem), or a younger family member's first big boy computer.
 >>/1448/
> Minis Forum
I recently picked up an Minis Forum HM80, and I like it a lot.  It's my first computer from this company, and it's my first desktop in about a decade.  I've been using laptops for so long before this, but this is a nice change of pace.  It's a lot faster than I thought it would be too.

I had a low productivity week, so I am going to skip the release and get more done on Wednesday.

v514 should be on the 25th of January. Thanks everyone!

----

For Client API developers, I am clearing out some obsolete calls and data structures in v514! Since we have a spare week, please review the pending breaking changes:

* '/add_tags/get_tag_services' is removed! use '/get_services' instead!
* 'hide_service_names_tags', previously made default true, is removed and its data structure 'service_names_to_statuses_to_display_tags' is also gone! move to the new 'tags' structure.
* 'hide_service_keys_tags' is now default true. it will be removed in 4 weeks or so. same deal as with 'service_names_to_statuses_to_display_tags'--move to 'tags'
* 'system_inbox' and 'system_archive' are removed from '/get_files/search_files'! just use 'system:inbox/archive' in the tags list

Full documentation preview, including some new commands and parameters, here:

https://hydrusnetwork.github.io/hydrus/developer_api_future.html

 >>/1449/
Yeah I almost feel like some part of them fell off the back of a truck, given what you spend for what you get. Even in OEM bulk, Windows Pro licenses aren't all that cheap.


I had a good couple of weeks working on a lot of different things. There's updated downloaders, a heap of bug fixes, optimisation and quality of life, a significant Client API update, and a new related tags method to test out.

The release should be as normal tomorrow. (actually later today)



https://youtube.com/watch?v=jkYg2eV3xBc
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v512/Hydrus.Network.512.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v512/Hydrus.Network.512.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v512/Hydrus.Network.512.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v512/Hydrus.Network.512.-.Linux.-.Executable.tar.gz

I had a great week back at work. There's a mix of different things, but particularly duplicate search improvements.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

duplicates

You can now set a search in the duplicates processing page for 'one file comes from search A, the other comes from search B'! It is now super easy to, say, find png vs jpeg pixel duplicates.

I also improved the overall accuracy and precision of the duplicate search logic. When I looked closer at this code, I wasn't always getting pixel duplicates or search distance correct when actually fetching results to show you, or I was selecting bad representatives of larger groups in the final step. This was particularly true in 'show some random potential pairs', which was showing too many files, often outside the search scope. There's some other bells and whistles that are subtle but should make the results feel more 'human'--for instance, in 'show some random potential pairs', if you are set to 'at least one file matches the search', then the first of the random group will definitely match the search, and if you are set to 'both files match different searches', then the first file matches the first and the others match the second.

I have made several changes here, sometimes significantly increasing the CPU load. In testing, things generally seemed to be running at ok speed, but let me know if your IRL complicated situation gets laggy. Also let me know if you discover any miscounts.

misc

When you select thumbnails or right-click on them, the labels in the menu and status bar are a bit nicer--it can now say 'x images' or 'x mp4s' instead of the flat 'x files' in more places, and any time you have a selection of files with duration, it will say the total duration.

v511's supersampled thumbnails now look good on any UI scale. If you have a multi-UI-scale setup, on multiple monitors or not, you should now set the new setting (under options->thumbnails) to your maximum UI scale and hydrus should look good everywhere!

If you want to import or export a .txt sidecar with a separator character other than newline (e.g. comma, for CSV format), you can now do this! You can set any separator characters you like.

The Client API can now refresh pages.

next week

Next week will be a medium-sized job week. I want to spend extra time adding the duplicates system to the Client API.
I had a good week doing one thing. The Client API now supports the main duplicates commands, fetching current file relationships and the main filter workflows, and then setting basic file relationships back again. This is a complicated feature for users who play around with the latest Client API tech, so tomorrow's release is for those advanced users only!

The release should be as normal tomorrow.



https://youtube.com/watch?v=_3ZC45Q82pg
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v511/Hydrus.Network.511.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v511/Hydrus.Network.511.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v511/Hydrus.Network.511.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v511/Hydrus.Network.511.-.Linux.-.Executable.tar.gz

𝕸𝖊𝖗𝖗𝖞 𝕮𝖍𝖗𝖎𝖘𝖙𝖒𝖆𝖘!

I had a good week fixing some things and adding some new options.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

First off, if you uploaded to v510 and had problems uploading to the PTR, sorry for the trouble! I missed something last week with an object update, and it impacted the network protocol. Thanks for the reports--I figured out a serverside patch and we updated the PTR.

I also fixed some logic in 'file import options' where it uses URLs and parsed hashes to predict if a file is 'already in db' or 'previously deleted' before a download. Some edge cases, particularly where many urls were mapped to one file, were not working quite right, and hashes tended to be overly dispositive. Ultimately, this thing should now be better at saving you bandwidth. The options here, which can be seen by advanced users, have updates too--check the changelog and tooltips for explanations.

If you have a large monitor and UI scale >100%, your thumbnails have looked bad for a while. Please check the new 'Thumbnail UI scale supersampling %' option under options->thumbnails. Set it to your monitor's UI scale, and your thumbnails should regen to be crisp. Let me know how it goes!

If you are into clever tag searching, check out the new 'unnamespaced input gives (any namespace) wildcard results' option under services->manage tag display and search. This is a first attempt to bring back the old 'unnamespaced search tags search namespaced variants' tech we used to have. It makes it so typing in an unnamespaced tag will give you the '*:subtag' variants quickly. These wildcards are labelled specially now, too.

next week

I am taking my Christmas vacation week! I'm going to spend some time with family and get lost in vidya. The week after, I'll see about sinking my teeth back into the serverside tag filter. v512 should be out January the 4th.

Thanks everyone!


 >>/1442/
I had this exact same thought over my vacation when I was looking at some vidya captures I had made. I'll add this!

 >>/1443/
Not yet, but I would like to add this. I've never really been happy with how hydrus handles 'page:1' style tags, but we'll never get better at it as long as it is difficult to easily add them to files after they are imported.

I had a great week back at work. I cleared a variety of small jobs and spent some extra time on the duplicates system. Several duplicates search bugs are fixed, and you can now set a potential pairs search for 'one file is in search A, the other is in search B'.

The release should be as normal tomorrow.



https://youtube.com/watch?v=81tEdqBvHVs
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v508/Hydrus.Network.508.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v508/Hydrus.Network.508.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v508/Hydrus.Network.508.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v508/Hydrus.Network.508.-.Linux.-.Executable.tar.gz

I had a great week fixing some bugs, adding hash lookup to the Client API, and finally attaching a Tag Filter to the PTR.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

tag filter

For normal users: the PTR will start filtering out some bad tags automatically, you don't have to do anything.

The big job I wanted to do this week, adding a Tag Filter to the PTR (and other tag repositories), went a lot better than I expected. I split the work into four pieces and got the first one done. If you are a tag repository administrator, you can now edit a Tag Filter for your repo under the services->administrate services->tag repo->edit tag filter menu. Your tag repo will silently discard any tag mapping uploads that do not pass the filter. So, if you don't want 'tagme' or 'source:some-borked-url' tags, you can now filter them at the point of upload and no one has to create or process any petitions in future.

Regular users will sync to the set filter and can see it under services->review services. Users in advanced mode will also get popups on specific rule changes. The next step here will be to expose this filter more, showing it in manage tags and similar UI, not even trying to pend tags that don't match the filter, and retroactively fixing mappings that already exist on the server (including hard-replacing since-sibling'd tags). I'd like to get at least some of this done before the end of the year, but I'm sure some will spill into 2023.

other highlights

There is a new 'file relationships: show x' action in the 'media' shortcut set, which will show a file's potential dupes, actual dupes, alternates, or false positives in a new page, just like the 'show x' actions that are buried in the thumbnail menu. This works in the media viewer too.

I fixed a recent stupid bug in the duplicate filter that meant if you did a batch of work with only file deletes, they were not going through. Sorry for the trouble! The thing now calculates and reports duplicate decisions and manual file deletes separately. I also added a (+50%, -33%) label to the file size comparison statement, just as a test. The numbers are correct, but I'm not sure I like it--let me know what you think.

If you work with the downloader system, URL Classes get two new checkboxes this week, 'do not allow any extra path components/parameters', which stop matching URLs that have too-long paths or parameters. It should help fix up some situations where hydrus was having trouble figuring out which URL Class to match sets of nested URLs to, particularly when the Gallery URL is 'longer' than the Post URL.

With the new /get_files/file_hashes call, the Client API can now do lookups between sha256, md5, sha1, and sha512 hashes. If you provide some hashes in one of these types, the API can now give you known matches for any of the other types. Let's say you have a bunch of sha1 hashes and want to see if you have them imported, or you have a bunch of sha256 hashes for files you have and want to get their known md5s for an external site lookup, this is now possible. The client typically knows all four hashes for every file it has ever imported, so you can do your own 'already deleted' checks too.

next week

More like this. Some small work and some more janitor/serverside stuff. Only three more releases this year, so I'll try to keep things simple.
I had a good week mostly fixing bugs, including the recent annoying-but-harmless shutdown bug some users had and the macOS app boot bug. There's also some UI quality of life.

The release should be as normal tomorrow.

 >>/1425/

Thanks for your continuing work on the project man, much appreciated. I have some feedback/wishes:

1. 4chan thread watcher via Tor gives 403 Cloudflare error - simple downloader works. Is there something one can do about this beyond switching circuits/proxy/clearnet? I mentioned this before and the browser header shouldn't make a difference here, as downloads as such work, just not the watcher.

2. 4chan simple downloader fails at .webms in same setup, these have to be manually added.

3. Add a second/third bar for tabs, so that the titles are not squeezed when having many tabs open.

4. PDF/ebook compatbility to use Hydrus for more informative tasks (this might get you a lot of support from librarian types).

5. Grouping of tags - as with the namespace/creator group by function, have colored bars or something that groups all items belonging to that category - or some similar function to have a "folder like" grouping in the main window. Tags are awesome for precise searches/classifications, but sometimes that "all in one" feeling that folders offer might be useful; I may have overlooked something though.

As always, I really love the work you do and you simplify a lot of time intensive stuff. I'd love to have a setup working that scrapes threads automatically by topic, I was thinking of a hydrus + something scripted setup that identifies threads by title/keyword and then automatically adds them to the watcher. This would save so much time and erase blind spots when covering news topics etc. Any ideas as to the feasibility?

 >>/1426/
Thanks, I'm glad you like it!

1) The watcher hits up the 4chan API (in this form https://a.4cdn.org/m/thread/16086187.json), whereas the simple downloader hits the main html. My guess is that's what CF is blocking. If you had to set up headers or User-Agent to get the html to work, might want to replicate it for a.4cdn.org. Hydrus Companion may help here to copy cookies, not sure. If CF just blocks API hits from TOR, you may be out of luck.

2) Yeah, I bet the simple downloader looks for img tags or something, whereas webms are under 'video' or an iframe or something. Simple downloader can't do two things at once very well, I'm afraid! If you are feeling brave, you could try editing the simple parsing formula to try to get both img and (video), but I'm not sure if it is that clever.

3) Check out the 'page of pages'. Hit F9->special->page of pages. That'll let you nest your page tabs.

4) Thanks. Yeah, I'd like some basic support for more text formats, and 'any file' support, which will take some extra work for some internal technical reasons. But I'd like it. However, I do not think I can support nice text reading any time soon. I'm terrible at drawing lines, but I know I'll never have the time to make something anywhere near as good as Calibre, so the support will stay as an 'open externally' button, like PDFs now, for a long time.

5) Yeah, I absolutely want [+] expand/tree format for the tag list. That list is my own custom widget, so I have to code everything in it. But this is something I really want for myself too, and options for things like 'start with title tags collapsed'. And custom namespace sorting so you can put creator up top, that sort of stuff.

Having auto-scraping has been a long-time thought. I'm hesitant to make too many automatic searchers, since I like human eyeballs behind big decisions. I know some users, however, who have had big success writing their own scripts that fetch the current thread catalog, scan it for threads they want, and then pass them on to Hydrus using the Client API. If you are ok with scripting, you might like to play around with it: https://hydrusnetwork.github.io/hydrus/client_api.html

As always, I'm short on work time. When I eventually roll some of 4 + 5 out, let me know how it goes for you, and if you have any other problems or ideas.

 >>/1428/
1) Will try that out, once I find the time. Thanks for the answer!

2) I'll experiment on that as well and will report back with how it goes.

3) Nice, awesome!

4) I'd be enough to merely have the works indexed inside Hydrus. You can always CTRL+E to open in your preferred reader/program with all the capabilities you need on the media itself. It's just so you can collect and organize the files, which Hydrus really blows anything else I know out of the water. There's some academic software that lets you organize your sources and stuff, but it's more of a list thing. Since you can add tons of metadata... well let's just say Hydrus and zlib would go hand in hand ;)

5) Yeah, collapsible tag trees would make stuff more easily readable. I know you can organize by custom namespaces and that is already really powerful, but it'd be cool to have tag:one tag:two tag:three below each other in the tag list and in the folder list as well, in "blocks" so to speak.

I feel you on automation. One misstep and you practically DDoS a site or flood your hard drive. I also didn't listen to your warning in the documentation haha. "and then THEY DO SO only to work through thousands of files", I felt that so hard and it brought a smile to my face. But once you have figured out what you want to parse daily etc. you then really want the automation for these specifica.

I really do appreciate your time man. Your project is one I always look into the documentation again and check the forums here. That's a rare exception for a really solid project. If you go the academic/library route, you might catch some licensing stuff to capitalize on your outstanding work, like for official use and stuff.



https://youtube.com/watch?v=8H40p44oo1U
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v510/Hydrus.Network.510.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v510/Hydrus.Network.510.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v510/Hydrus.Network.510.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v510/Hydrus.Network.510.-.Linux.-.Executable.tar.gz

I had a good week improving notes and UI quality of life. It is Hydrus's 11th birthday today!

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

File notes got several updates this week, mostly for copying notes from one file to another. First off, the 'duplicate metadata merge options' finally support notes! All users will get some basic 'yes please, merge my notes' defaults set, copying from worst to best and copying both ways when the files are the same quality. If you do a lot of duplicate filtering and have been waiting for this, double check your options (under the duplicates processing page) to make sure you are happy. Relatedly, the notes dialog now has copy/paste buttons for copying from one file to another. To get this all to work, these actions use some Note Import Options tech to handle conflicts, generally by extending existing sub-notes where possible and renaming on conflicts. If you want to work with notes in the Client API, this 'easy merge' tech is also optionally available in the Client API. Let me know how this works for you--I think I am going to have to adjust the logic to handle two-way related note merges a little better, so feedback on IRL situations would be great.

I significantly improved how the taglist handles shift-selections and drag-selections. A bunch of position and logic jank is cleaned up, and now you can 'undo' a multi-selection, just like I did for thumbnails recently. Best way to test this is just click and drag down the list, and then move your mouse up--it undoes the selection as you move up, and it remembers what was originally selected before! Also, holding ctrl down to start a deselection drag is cleaner and more human. Give it a go, and you'll feel the difference.

next week

Just one more week of work and then I am on my vacation. I'll do some more cleanup/small jobs and try to catch up on github bug reports, which I have been neglecting.
birthday

The first non-experimental beta of hydrus was released on December 14th, 2011. We are now going on eleven years.

I had a challenging 2022. Several IRL problems appeared from nowhere, but I decided to keep on trudging, and we got some work done regardless. I'm satisfied with the result. We started the year with the run up to multiple local file services, a long-planned expansion to the sheer shape of the database, and with that, beyond the new file services, came an array of new database search tech, from the ability to search deleted files and search unions of different file domains to enhanced cancel tech for snappier UI and more accurate tag counts and the nice user-made file history chart. We also saw new time search tech, with nicer UI, the ability to search the last view time of a file, the recording of archive times, and the broad 'modified time' parsing expanson, which combines post times from many sources to give you a more precise 'source time' of the things you download. Thanks to user help, we also figured out the new help, which looks great, is easily searchable, and is simple to edit. We hammered out more stable versions of mpv and other spicy libraries for the different builds and user environments. The UI had numerous quality of life improvements, from dejanked layouts and sizing calculations to fixed custom widgets and overhauled hover windows to cleaner selection logic and quick-action buttons for easy pastes and mass-application actions. File notes are finally parsable and now display on the media viewer. Sidecar importing and exporting became more powerful. EXIF and other human-readable file metadata is now parsed and viewable. The serverside janitor workflow finally received some improvement. The Client API grew to support autocomplete tag searching, note editing, cleverer file reference and file delete, nicer thumbnail fetching, better error/disconnect handling, tag display type support, more hash fetchings and lookups, and the new timestamps and file metadata statuses like EXIF. And the program became significantly easier for any user to run from source.

There is still no end of work to do, and I am thankfully able to continue, so I plan to keep going into 2023. I deeply appreciate the feedback, help, and support over the years. Thank you!

If you would like to further support my work and are in a position to do so, my simple no-reward Patreon is here: https://www.patreon.com/hydrus_dev

Congrats on turning 11, dev. Hope you're doing well.
I've been using Hydrus off and on since 2015, but I reimported a few times early on so my oldest files are from 6 years ago. I've been going through some of them, finally getting around to sorting a bunch of stuff I scraped from tumblr back then.
I just transferred my "active" copy of Hydrus to a new drive, figured the original one might be getting worn out, best to keep it as the backup. In the process, it seems like it forgot to keep sound off, that or sound in-client is just a recent addition.
Is there a way I can toggle sound on and off globally for the program?

I had a good week. I fixed some bugs, added some new import and tag search options for advanced users, and made thumbnails look crisp at high UI scale.

The release should be as normal tomorrow.

 >>/1437/
Thanks. Yeah, I am doing good overall. Looking forward to my vacation next week though!

We added 'mpv' support I think two or three years ago--that's where the sound is coming from. You should notice video plays a lot smoother too. Any time you are looking at a noisy video, move your mouse to the bottom of the video frame so the scanbar pops up. On the right should be a 'speaker' symbol. Click it and you'll get global mute. There's another speaker symbol in a popup that just mutes preview/media viewer.

There's some shortcuts for this too, under file->shortcuts->global.




Post(s) action:


Moderation Help
Scope:
Duration: Days

Ban Type:


0 replies | 0 file
New Thread
Max 20 files0 B total
Refresh