Update on this: I got the feature work done this weekend, so now I'll be testing it a bunch for upgrades and storage migrations
I am aware of garage, but haven't tested it yet with pict-rs. It's a cool project for sure
Yes. It uses sha256 rather than perceptual hashing, but that's Good Enoughβ’οΈ
I'm not against including an ipfs layer in pict-rs, but the complexity would go way up. Federating an image between lemmy servers would require sending the ipfs uri between servers via activitypub, and then each receiving server sending that uri to pict-rs. pict-rs would then need to decide, on each server, if the ipfs-stored image matches that servers' image requirements (max filesize, max dimensions, etc), and if it does, then that pict-rs server would request to pin the image. I don't know exactly how ipfs pinning works, but ideally it would only be stored locally if it isn't already stored in at least N other locations. If the remote image doesn't match the local server's configuration, it could either be rejected or downloaded & processed (resized etc).
Serving ipfs-stored images that aren't replicated locally might also be slow, but I won't know for sure unless I actually try building this out.
I can only say "when it's ready." I think most of what I want to include in 0.4 is there, but I don't have a ton of time to work on it currently. I might see if I can get my last feature changes in this weekend, then it will be a matter of ensuring the 0.3 -> 0.4 upgrade is smooth, and that storage migration is solid
pict-rs doesn't keep track of how often it serves different images, so there's not a good metric for pruning old images. That said, 0.4 will introduce functionality for cleaning up processed images (e.g. resizes/thumbnails), removing their files & metadata. If they are viewed again, they will be re-generated.
0.4 will also include the ability to scale down images on upload, rather than storing the original resolution. This is not yet implemented, but it's on my roadmap.
All this said, it is already possible to use pict-rs with object storage (s3-compatible), rather than block storage. That's a good option if your hosting provider offers it
what I noticed personally is that my pict-rs server had eaten 1,200MB of RAM, which is the limit I allowed that container. it was still working and it automatically restarted on crash so I didn't notice it oops
after I fixed the problem, my pict-rs server has been running at about 29MB for the last 12 hours. Significant improvement I'd say
for context: I am the pict-rs developer
Problem:
I gave tokio-console a buffer way too big
Fix:
Update pict-rs to at least v0.3.0-rc.7, which doesn't enable console by default
how long does it take for pict-rs to use up this memory?
If you don't know what OpenTelemetry is but you're having OpenTelemetry errors in your logs, it likely means that you have an opentelemetry_url set in your config when you shouldn't. You can get rid of the error by deleting that line, which will disable opentelemetry exports
view more: next βΊ
I chose it at the start of the project π€·