/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.


New Reply on thread #1244
X
Max 20 files0 B total
[New Reply]

[Index] [Catalog] [Banners] [Logs]
Posting mode: Reply [Return]


https://youtube.com/watch?v=tigTaObQORM
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v476/Hydrus.Network.476.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v476/Hydrus.Network.476.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v476/Hydrus.Network.476.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v476/Hydrus.Network.476.-.Linux.-.Executable.tar.gz

I had a good week integrating two new features: autocomplete tag search in the client api, and saved 'post times' from downloaders.

post times

'Modified time' is neat, but it isn't super useful for downloaded files--since the file was only just added to your hard drive, it'll always be the same as import time. This week I integrate the 'source time' we parse from various websites to improve the modified time for downloaded files. The objective is to make 'modified' a fairly decent 'this file was completed around this time' number for searching and sorting purposes.

I'm being careful not to overwrite anything. The client now saves its best 'source time' for every different site it downloads from and then the earliest of those + modified date is used as the aggregate modified date. You don't have to do anything, but with luck you will see your new watcher and gallery files start to get some nicer modified times in the media viewer and thumbnail right-click menu.

There are many potential future expansions here. I can grab better post times from sites, show and edit every stored timestamp in UI, allow clever search and sort of those specifically (e.g. 'sort all these files by their danbooru post time), and most importantly make some sort of maintenance system to retroactively fetch a good post time for all the files we downloaded before post times were saved. This is just a first step.

I integrated the new 'archive time' too during this work. This now shows in the media viewer and thumbnail right-click menu similarly and can be sorted by. Search will come soon. I also want to think about optionally filling in some estimate dummy data here for all the files we archived before timestamps were tracked.

client api autocomplete search

A user has helpfully written Client API routines for autocomplete tag search, which is something I have had trouble fitting in. I appreciate the work. This should let the various tools that use the Client API do more tag browsing in future.

The documentation is here: https://hydrusnetwork.github.io/hydrus/developer_api.html#add_tags_search_tags

There are several ways to expand this too, so if you are an API dev interested in it, let me know how it goes.
full list

- domain modified times
- the downloader now saves the 'source time' (or, if none was parsed, 'creation time') for each file import object to the database when a file import is completed. separate timestamps are tracked for every domain you download from, and a file's number can update to an earlier time if a new one comes in for that domain
- I overhauled how hydrus stores timestamps in each media object and added these domain timestamps to it. now, when you see 'modified time', it is the minimum of the file modified time and all recorded domain modified times. this aggregated modfified time works for sort in UI and when sorting before applying system:limit, and it also works for system:modified time search. the search may be slow in some situations--let me know
- I also added the very recent 'archived' timestamps into this new object and added sort for archived time too. 'archived 3 minutes ago' style text will appear in thumbnail right-click menus and the media viewer top status text
- in future, I will add search for archive time; more display, search, and sort for modified time (for specific domains); and also figure out a dialog so you can manually edit these timestamps in case of problems
- I also expect to write an optional 'fill in dummy data' routine for the archived timestamps for files archived before I started tracking these timestamps. something like 'for all archived files, put in an archive time 20% between import time and now', but maybe there is a better way of doing it, let me know if you have any ideas. we'll only get one shot at this, so maybe we can do a better estimate with closer analysis
- in the longer future, I expect import/export support for this data and maintenance routines to retroactively populate the domain data based on hitting up known urls again, so all us long-time users can backfill in nicer post times for all our downloaded files
- .
- searching tags on client api:
- a user has helped me out by writing autocomplete tag search for the client api, under /add_tags/search_tags. I normally do not accept pull requests like this, but the guy did a great job and I have not been able to fit this in myself despite wanting it a lot
- I added some bells and whistles--py 3.8 support, tag sorting, filtering results according to any api permissions, and some unit tests
- at the moment, it searches the 'storage' domain that you see in a manage tags dialog, i.e. without siblings collapsed. I can and will expand it to support more options in future. please give it a go and let me know what you think
- client api version is now 26
- .
- misc
- when you edit something in a multi-column list, I think I have updated every single one so the selection is preserved through the edit. annoyingly and confusingly on most of the old lists, for instance subscriptions, the 'ghost' of the selection focus would bump up one position after an edit. now it should stay the same even if you rename etc... and if you have multiple selected/edited
- I _think_ I fixed a bug in the selected files taglist where, in some combination of changing the tag service of the page and then loading up a favourite search, the taglist could get stuck on the previous tag domain. typically this would look as if the page's taglist had nothing in it no matter what files were selected

- if you set some files as 'alternates' when they are already 'duplicates', this now works (previously it did nothing). the non-kings of the group will be extracted from the duplicate group and applied as new alts
- added a 'BUGFIX' checkbox to 'gui pages' options page that forces a 'hide page' signal to the current page when creating a new page. we'll see if this patches a weird error or if more work is needed
- added some protections against viewing files when the image/video file has (incorrectly) 0 width or height
- added support for viewing non-image/video files in the duplicate filter. there are advanced ways to get unusual files in here, and until now a pdf or something would throw an error about having 0 width

next week

Back to multiple local file services, which is in endgame. I have a ton of ancient file handling code to simply clean to newer standards.

I had a good week. As well as some general fixes and more file modified time work, there is also a user-written expansion to the Client API that adds 'notes' editing support and a bit of fun.

The release should be as normal tomorrow.


Post(s) action:


Moderation Help
Scope:
Duration: Days

Ban Type:


3 replies | 0 file
New Reply on thread #1244
Max 20 files0 B total