/hydrus/ - Hydrus Network

full list

- misc:
- fixed the simple delete files dialog for trashed files. due to a logical oversight, the simple version was not testing 'trashed' status and so didn't see anything to permanently delete and would immediately dump out. now it shows the option for trashed files again, and if the selection includes trash and non-trash, it shows multiple options
- fixed an error in the 'show next pair' logic of the new duplicate filter queue where if it needed to auto-skip through the end of the current batch and load up the next batch (issues #1139, #1143)
- a new setting on _options->media_ now lets you set the scanbar to be small and simple instead of hidden when the mouse is moved away. I liked this so much personally it is now the default for all users. try it out!
- the media viewer's taglist hover window will now never send a mouse wheel event up to the media viewer canvas (so scrolling the tags won't accidentally do previous/next if you hit the end of the list scrollbar)
- I think I have fixed the bug where going on the media viewer from borderless fullscreen to a regular window would not trigger a media container resize if the media perfectly fitted the ratio of the fullscreen monitor!
- the system tray icon now has minimise/restore entries
- to reduce confusion, when a content parser vetoes, it now prepends the file import 'note' with 'veto: '
- the 'clear service info cache' job under _database->regenerate_ is renamed to 'service info numbers' and now has a service selector so you can, let's say, regen your miscounted 'number of files in trash' count without triggering a complete recount of every single mapping on the PTR the next time you open review services
- hydrus now recognises most (and maybe all) windows executables so it can discard them from imports confidently. a user discovered an interesting exe with embedded audio that ffmpeg was seeing as an mp3--this no longer occurs
- the 'edit string conversion step' dialog now saves a new default (which is used on 'add' events) every time you ok it. 'append extra text' is no longer the universal default!
- the 'edit tag rule' dialog in the parsing system now starts with the tag name field focused
- updated 'getting started/installing' help to talk more about mpv on Linux. the 'libgmodule' problem _seems_ to have a solid fix now, which is properly written out there. thanks to the users who figured all this out and provided feedback
- .
- multiple local file services:
- the media viewer menu now offers add/move actions just like the thumb grid
- added a new shortcut action that lets you specify add/move jobs. it is available in the media shortcut set and will work in the thumbnail grid and the media viewer
- add/move is now nicer in edge cases. files are filtered better to ensure only local media files end up in a job (e.g. if you were to try to move files out of the repository update domain using a shortcut), and 'add' commands from trashed files are naturally and silently converted to a pure undelete

- boring code cleanup:
- refactored the UI side of multiple local file services add/move commands. various functions to select, filter, and question the user on actions are now pulled to a separate simple module where other parts of the UI can also access them, and there is now just one isolated pipeline for file service add/move content updates.
- if a 'move' job is started without a source service and multiple services could apply, the main routine will now ask the user which to use using a selector that shows how many files each choice will affect
- also rewrote the add/move menu population code, fixed a couple little issues, and refactored it to a module the media viewer canvas can use
- wrote a new menu builder that can place a list of items either as a single item (if the list is length 1), or make a submenu if there are more. it drives the new add/move commands and now the behind the scenes of all other service-based menu population

next week

Next week is a cleanup week, so I will do some boring code cleanup and see if I can write some nice introductory help for the multiple local file services system. I have four more weeks before my vacation, so I am aiming to have the big work of multiple local file services finished by then.

 >>/1280/
I'm on version 482 and got some kind of error message on the bottom right hand side of the client but I accidentally right clicked and closed out of the message so I don't know what it was about. Is there a way to view past logs? I don't want to update until I know what this was.

 >>/1284/
There are log files with full records in your 'database' directory, which by default is install_dir/db, but if you hit file->open in the client, it can take you straight to it. They are called 'client - year-month.log'.

Scroll down to the bottom of the log to see the latest messages. If you ctrl+f for 'traceback' (maybe 'up' from the bottom), that may help you find the error.

 >>/1285/
Thanks I found it.

v482, 2022/05/14 06:18:17: A thumbnail error has occurred. The problem thumbnail will appear with the default 'hydrus' symbol. You may need to take hard drive recovery actions, and if the error is not obviously fixable, you can contact hydrus dev for additional help. Specific information for this first error follows. Subsequent thumbnail errors in this session will be silently printed to the log.



The thumbnail for file d9e0473a95799d103ef745c52eb8803caa4326947f34bba98efaf1eac8ebdf63 could not be regenerated from the original file because the original file is missing! This event could indicate hard drive corruption. Please check everything is ok.

Yeesh, I regret wanting to know this. How do I check if everything is ok?

 >>/1286/
To add more information, I run Hydrus on an external HDD. I used crystaldisk to check the state of the drive and everything seems ok, no bad signs. It just seems like a file went corrupt and went missing from my database in the folder. The file was imported 19 days ago from now but not archived, I'm not sure if being archived does anything but I'll be on the lookout for any more issues.

I had a good simple week. I cleaned some code, improved some quality of life, and made multiple local file services ready for all users.

The release should be as normal tomorrow.

 >>/1286/
 >>/1287/
Hmm, this is a shame. I am not sure what happened here. The most likely culprit is a hard drive fault, although those tend to make a file 0KB more often than simply disappearing a file. Sometimes you can have anti-virus go bananas and remove an mp3 or something, but this seems to be just a jpeg.

If you are worried this may have happened to more files, the place to check is database->file maintenance->manage scheduled jobs. Under 'add new work', set up a job for 'all media files' for an appropriate job type. The default, 'if file missing, try to redownload, else remove record' is good.

If it is any help, I don't have that file, but I think I found it through its PTR tags here:

https://danbooru.donmai.us/posts/5084454

You can probably see that on the borked thumbnail's right-click menu too. Just enter that URL into a new URL downloader page and fingers crossed hydrus should be able to automatically fix the file for you. The above maintenance job will do this automatically.

Let me know if you have any more trouble, and make sure you have a good backup!
https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#backing_up

57457547 jpg
(175.59 KB, 1124x690)

 >>/1288/
Thanks, I encountered a few more. Same day from that same session. I'm not sure what happen that day, I don't think I badly closed out of the program or improperly dismounted the drive. It might be that I haven't been doing regular maintenance other than shutting down and I definitely haven't been defraging the drive like I probably should so its probably that and might be bad sectors or something.

Also the other few I mentioned I encountered gave me a slightly different message. I can view the thumbnail but whenever I tried to open it it gave me this message. It would then redownloaded the file but still wouldn't load it. I forgot to check the actual folder where the file is located and just set my HDD aside until I got a proper solution.

 >>/1289/

Ah, for files that seem to be malformed, there are two answers:

1) The file might have been corrupted. In the 'file maintenance' system, there's a set of jobs called 'missing/incorrect' that scan for this. These jobs are expensive though (they have to load the whole file from disk), so don't add them for all files. Try to target with a tag search or system:hash search just to hit the file(s) you think were affected.

2) The file might have once been supported but now is not. This is rare, but a handful of truncated jpegs and other borked files used to load in hydrus until I turned some safety rules on (the lack of safety was causing crashes in some systems). Best answer here is to just delete the file, or export it, run it through a load/save cycle in Gimp or Photoshop, and then reimport the fixed file. Some boorus just have jank files unfortunately.

If you like, you can post that image here (if the site can accept it), or zip it up and email it to me, and I can have a look. But if it is private, no worries.

These bad files being from the same session makes sense that this was a one-time event. I would still suspect it is a hard drive issue, but it could be something else, especially since crystaldiskinfo was good for you. There's a very small chance that if you have a program crash or other hard turn-off that the client can become unsynced, but that typically ends up being the other way around: the file gets added to your hydrus file storage, but the client isn't aware of it. The client can lose up to 30 seconds of database work if it ends non-cleanly.

Let me know how you get on!

 >>/1290/
I ran some checks and found no more issues. I did the "if the file is missing/incorrect" option. It seems like it was just 2 sessions from that day acting up. 1 tag session entry gave me 2 missing file errors and the other entry session gave me 1 corrupt file error. Maybe it was my anti-virus acting up and doing scans over the files I was just viewing. Those were just normal jpgs and png images, nothing really out of the ordinary.


I'll probably be more cautious and do backups just in case. I already have a NAS but never used it and don't know how I should set it up with hydrus but I'll probably make a new thread here(or somewhere more active) to discuss storage backups and safety. Thanks again.