Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Quick Links


Shells Virtual Desktop
BMail.ag - Secure Email Service
Server.net
CPLicense.net
VPS Server
Buy VPN
Vultr
VMs for AI
HostDare
ReliableSite White-Label Dedicated Hosting for Resellers
InterServer VPS
BMail.ag - Secure Email Service
Best VPN
High-Performance Bare Metal Server Solutions
Karvl.com
Server Mania Cloud Hosting
DataWagon Hosting
AlphaVPS Hosting
Evoxt.com
Clouvider
VPS Hosting with NVMe
Residential IPs in the US & 4G Mobile Proxies in EU & US with Unlimited Bandwidth
ReliableSite White-Label Dedicated Hosting for Resellers
Rabisu - Hosting Solutions
Shells Virtual Desktop

Categories

In this Discussion

Home › Offers
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

SPANKING NEW DEALS! HAPPY NEW YEAR w/ PREMIUM GIVEAWAYS! FREEBIES GALORE by RackNerd.

1…470471473475476…1047

Comments

  • BeniskickbuttBeniskickbutt Member

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    Sure, dude. Stay active and you won't regret it.

    Yeah, lets see btw why is dustinc not visible didn't see him yesterday and even today too

    Let's not forget the he's the CEO of RackNerd and that it's still the holiday season, being one of the busiest times for them. So, he might be extremely busy. But rest assured. He will be here with many GAs. Hope you can hang around till and each time it happens.

    Yeah I'm surprised he even has as much time as he does to drop in here

  • DeusVultDeusVult Member
    edited December 2024

    That's my aim

  • noob404noob404 Member

    @Beniskickbutt said:
    Oh wow you guys been busy today 🙂

    Covered some great topics. Since youa re here, let us choose a new topic, if you want

  • BeniskickbuttBeniskickbutt Member

    @TrK said:

    @noob404 said:

    @TrK said:

    @noob404 said:

    @TrK said:
    another random suggestion: Don't compile mariadb 11.4 on a slow VPS.... it will take hours to finish the task. or just compile it and go back to sleep... it will be done when you wake up... probably....

    Remember trying to compile PHP. Afew years ago on a 2GB VPS. Made me wait for 2 hours and then said you need atleast 4GB RAM! :lol:

    well different cases for different tasks you can bypass the ram requirements with swap but i don't usually wait more than an hour for my compilation tasks.

    Oh yes. Infact there was a workaround (non-swap) for building it that I learned of back then. Don't remember now. I prefer my personal PC for compilation.

    You mean the raw power you have locally :smile:

    That lappy will only help his compiles

  • noob404noob404 Member

    @TrK said:

    @cainyxues said:

    @TrK said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:
    My vps is basically idle at the moment.

    The only VPS you have?

    Yes

    Waiting for a good project to make good use of the VPS?

    Yeah. I just don't know what to code.
    My wayback machine project is fun, but if i would actually deploy it, it would require a lot of storage. And i primarily make it to learn more about web crawling and puppeteer.

    Oh that's cool. Yah, wayback machine would require a hell lot of storage.

    Well one thing that is different than the WA, is that i only save full-page screenshots instead of the whole page with all the assets.

    Oh so a waybackscreenshots. Sounds coolm.in that case you could compress the images using various tools. Infact, @TrK is working on a commercial image optimisation CDN that might be of help for your project

    or if you have an idle vm in your inventory you can always use imgproxy :wink:

    is this your commercial project ?

    nope it's an open source project, i used for quite some time :smile:

    Youa re gonna open source it?

  • BasToTheMaxBasToTheMax Member

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:
    My vps is basically idle at the moment.

    The only VPS you have?

    Yes

    Waiting for a good project to make good use of the VPS?

    Yeah. I just don't know what to code.
    My wayback machine project is fun, but if i would actually deploy it, it would require a lot of storage. And i primarily make it to learn more about web crawling and puppeteer.

    Oh that's cool. Yah, wayback machine would require a hell lot of storage.

    Well one thing that is different than the WA, is that i only save full-page screenshots instead of the whole page with all the assets.

    Oh so a waybackscreenshots. Sounds coolm.in that case you could compress the images using various tools. Infact, @TrK is working on a commercial image optimisation CDN that might be of help for your project

    Yeah average image size is currently around 25-100 kb. Small pages are around 25-50 kb, while large pages can easily reach 300 kb.

    You can bring them down a bit more.

    How? I am currently using the avif image format.

    I am not sure right now. But, I did work on this a few years ago. BTW, how are you compressing currently?

    First, the page screenshot is using png, then i compress to webp and then i compress that webp image to avif, but this only saves like 5-30 kb at the moment. Also the avif compression takes like 30 seconds.
    I am using the nodejs sharp library. I will send my code soon.

    That's quite impressive. Just to test though, try running the original image through tinypng.com and see if you are already able to get it a lower size than your final compressed image, if you haven't already. Just to confirm.

    Will try that. Don't have my laptop with me at the moment.

  • noob404noob404 Member

    @Beniskickbutt said:

    @noob404 said:

    @TrK said:

    @noob404 said:

    @TrK said:
    another random suggestion: Don't compile mariadb 11.4 on a slow VPS.... it will take hours to finish the task. or just compile it and go back to sleep... it will be done when you wake up... probably....

    Remember trying to compile PHP. Afew years ago on a 2GB VPS. Made me wait for 2 hours and then said you need atleast 4GB RAM! :lol:

    well different cases for different tasks you can bypass the ram requirements with swap but i don't usually wait more than an hour for my compilation tasks.

    Oh yes. Infact there was a workaround (non-swap) for building it that I learned of back then. Don't remember now. I prefer my personal PC for compilation.

    That's what I usually try to do as well. Local compile remote deploy

    My preferred method as well. Somehow I feel I got more control when it's local.

  • cainyxuescainyxues Member

    @TrK said:

    @cainyxues said:

    @TrK said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:
    My vps is basically idle at the moment.

    The only VPS you have?

    Yes

    Waiting for a good project to make good use of the VPS?

    Yeah. I just don't know what to code.
    My wayback machine project is fun, but if i would actually deploy it, it would require a lot of storage. And i primarily make it to learn more about web crawling and puppeteer.

    Oh that's cool. Yah, wayback machine would require a hell lot of storage.

    Well one thing that is different than the WA, is that i only save full-page screenshots instead of the whole page with all the assets.

    Oh so a waybackscreenshots. Sounds coolm.in that case you could compress the images using various tools. Infact, @TrK is working on a commercial image optimisation CDN that might be of help for your project

    or if you have an idle vm in your inventory you can always use imgproxy :wink:

    is this your commercial project ?

    nope it's an open source project, i used for quite some time :smile:

    So what was the image CDN commercial project that noob was talking about ?

  • xpress7xpress7 Member

    Lots of pages.. dont' want to go back now. Whats the topic currently? :D

  • DeusVultDeusVult Member

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    You will probably win a prize imo. Everyone here will. But the value of the prize depends on your own work

  • BeniskickbuttBeniskickbutt Member

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @TrK said:
    another random suggestion: Don't compile mariadb 11.4 on a slow VPS.... it will take hours to finish the task. or just compile it and go back to sleep... it will be done when you wake up... probably....

    Remember trying to compile PHP. Afew years ago on a 2GB VPS. Made me wait for 2 hours and then said you need atleast 4GB RAM! :lol:

    lol :lol:

    Oh that was a disaster. To overcome this, I tried compiling on my PC and moving it to the VPS. only after I installed it did I know that the specific module for which I rebuilt PHP won't work without a few more dependencies. Learned akt that day

    akt means?

    Pardon I am on my phone. I meant "a lot"

    Your English is pretty good. You mentioned wanting to get better. A lot of people use alot instead

    Thanked by 1noob404
  • DeusVultDeusVult Member

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    Sure, dude. Stay active and you won't regret it.

    Yeah, lets see btw why is dustinc not visible didn't see him yesterday and even today too

    I guess he's on holidays

  • noob404noob404 Member

    @Beniskickbutt said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    Sure, dude. Stay active and you won't regret it.

    Yeah, lets see btw why is dustinc not visible didn't see him yesterday and even today too

    Let's not forget the he's the CEO of RackNerd and that it's still the holiday season, being one of the busiest times for them. So, he might be extremely busy. But rest assured. He will be here with many GAs. Hope you can hang around till and each time it happens.

    Yeah I'm surprised he even has as much time as he does to drop in here

    Yah, he has some great time management skills

  • noob404noob404 Member

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:
    My vps is basically idle at the moment.

    The only VPS you have?

    Yes

    Waiting for a good project to make good use of the VPS?

    Yeah. I just don't know what to code.
    My wayback machine project is fun, but if i would actually deploy it, it would require a lot of storage. And i primarily make it to learn more about web crawling and puppeteer.

    Oh that's cool. Yah, wayback machine would require a hell lot of storage.

    Well one thing that is different than the WA, is that i only save full-page screenshots instead of the whole page with all the assets.

    Oh so a waybackscreenshots. Sounds coolm.in that case you could compress the images using various tools. Infact, @TrK is working on a commercial image optimisation CDN that might be of help for your project

    Yeah average image size is currently around 25-100 kb. Small pages are around 25-50 kb, while large pages can easily reach 300 kb.

    You can bring them down a bit more.

    How? I am currently using the avif image format.

    I am not sure right now. But, I did work on this a few years ago. BTW, how are you compressing currently?

    First, the page screenshot is using png, then i compress to webp and then i compress that webp image to avif, but this only saves like 5-30 kb at the moment. Also the avif compression takes like 30 seconds.
    I am using the nodejs sharp library. I will send my code soon.

    That's quite impressive. Just to test though, try running the original image through tinypng.com and see if you are already able to get it a lower size than your final compressed image, if you haven't already. Just to confirm.

    Will try that. Don't have my laptop with me at the moment.

    Sure, let me know when you have. I am interested in this as well

  • DeusVultDeusVult Member

    @TrK said:

    @Beniskickbutt said: Whats just another idle vm 😀

    i mean everyone gets one or two idles in the portfolio :wink:

    I have a IPV6 VPS that I idle because I don't have IPV6 sadly...

  • noob404noob404 Member

    All the talk of local compilation just reminded me, I once compiled Batocera from source on my i5-5250u. Took 5 days!

  • cainyxuescainyxues Member

    @DeusVult said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    You will probably win a prize imo. Everyone here will. But the value of the prize depends on your own work

    Lets see and just hope for the best for everyone

  • DeusVultDeusVult Member

    @Beniskickbutt said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    Sure, dude. Stay active and you won't regret it.

    Yeah, lets see btw why is dustinc not visible didn't see him yesterday and even today too

    Let's not forget the he's the CEO of RackNerd and that it's still the holiday season, being one of the busiest times for them. So, he might be extremely busy. But rest assured. He will be here with many GAs. Hope you can hang around till and each time it happens.

    Yeah I'm surprised he even has as much time as he does to drop in here

    Same. Initially I thought it was his team, but it's him.

  • BeniskickbuttBeniskickbutt Member

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    Sure, dude. Stay active and you won't regret it.

    Yeah, lets see btw why is dustinc not visible didn't see him yesterday and even today too

    Let's not forget the he's the CEO of RackNerd and that it's still the holiday season, being one of the busiest times for them. So, he might be extremely busy. But rest assured. He will be here with many GAs. Hope you can hang around till and each time it happens.

    Yeah man I completely forgot about it, It was just that he responds on each and every thread so that's why I was curious and got my answers too.

    Part of being CEO i guess. Doe a little of it all including marketing and support

  • noob404noob404 Member

    @xpress7 said:
    Lots of pages.. dont' want to go back now. Whats the topic currently? :D

    Img compression best methods. @BasToTheMax wants to know

    Thanked by 1BasToTheMax
  • DeusVultDeusVult Member

    @cainyxues said:

    @DeusVult said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    You will probably win a prize imo. Everyone here will. But the value of the prize depends on your own work

    Lets see and just hope for the best for everyone

    Yes!!

    Thanked by 1cainyxues
  • noob404noob404 Member

    @Beniskickbutt said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @TrK said:
    another random suggestion: Don't compile mariadb 11.4 on a slow VPS.... it will take hours to finish the task. or just compile it and go back to sleep... it will be done when you wake up... probably....

    Remember trying to compile PHP. Afew years ago on a 2GB VPS. Made me wait for 2 hours and then said you need atleast 4GB RAM! :lol:

    lol :lol:

    Oh that was a disaster. To overcome this, I tried compiling on my PC and moving it to the VPS. only after I installed it did I know that the specific module for which I rebuilt PHP won't work without a few more dependencies. Learned akt that day

    akt means?

    Pardon I am on my phone. I meant "a lot"

    Your English is pretty good. You mentioned wanting to get better. A lot of people use alot instead

    We have been taught to use both depending on the sentence

  • DeusVultDeusVult Member

    @Beniskickbutt said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    Sure, dude. Stay active and you won't regret it.

    Yeah, lets see btw why is dustinc not visible didn't see him yesterday and even today too

    Let's not forget the he's the CEO of RackNerd and that it's still the holiday season, being one of the busiest times for them. So, he might be extremely busy. But rest assured. He will be here with many GAs. Hope you can hang around till and each time it happens.

    Yeah man I completely forgot about it, It was just that he responds on each and every thread so that's why I was curious and got my answers too.

    Part of being CEO i guess. Doe a little of it all including marketing and support

    He's the OG

  • xpress7xpress7 Member

    @noob404 said:

    @xpress7 said:
    Lots of pages.. dont' want to go back now. Whats the topic currently? :D

    Img compression best methods. @BasToTheMax wants to know

    TrK knows better. He is building one optimization service.

  • BasToTheMaxBasToTheMax Member

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:
    My vps is basically idle at the moment.

    The only VPS you have?

    Yes

    Waiting for a good project to make good use of the VPS?

    Yeah. I just don't know what to code.
    My wayback machine project is fun, but if i would actually deploy it, it would require a lot of storage. And i primarily make it to learn more about web crawling and puppeteer.

    Oh that's cool. Yah, wayback machine would require a hell lot of storage.

    Well one thing that is different than the WA, is that i only save full-page screenshots instead of the whole page with all the assets.

    Oh so a waybackscreenshots. Sounds coolm.in that case you could compress the images using various tools. Infact, @TrK is working on a commercial image optimisation CDN that might be of help for your project

    Yeah average image size is currently around 25-100 kb. Small pages are around 25-50 kb, while large pages can easily reach 300 kb.

    You can bring them down a bit more.

    How? I am currently using the avif image format.

    I am not sure right now. But, I did work on this a few years ago. BTW, how are you compressing currently?

    First, the page screenshot is using png, then i compress to webp and then i compress that webp image to avif, but this only saves like 5-30 kb at the moment. Also the avif compression takes like 30 seconds.
    I am using the nodejs sharp library. I will send my code soon.

    That's quite impressive. Just to test though, try running the original image through tinypng.com and see if you are already able to get it a lower size than your final compressed image, if you haven't already. Just to confirm.

    Will try that. Don't have my laptop with me at the moment.

    Sure, let me know when you have. I am interested in this as well

    Oh by the way, the crawler already has added 800k urls in the db. About 5 of them are actually screenshotted (as that part is still work in progress).

  • cainyxuescainyxues Member

    @DeusVult said:

    @TrK said:

    @Beniskickbutt said: Whats just another idle vm 😀

    i mean everyone gets one or two idles in the portfolio :wink:

    I have a IPV6 VPS that I idle because I don't have IPV6 sadly...

    which one ? specs and provider..............?

  • noob404noob404 Member

    @Beniskickbutt said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    Sure, dude. Stay active and you won't regret it.

    Yeah, lets see btw why is dustinc not visible didn't see him yesterday and even today too

    Let's not forget the he's the CEO of RackNerd and that it's still the holiday season, being one of the busiest times for them. So, he might be extremely busy. But rest assured. He will be here with many GAs. Hope you can hang around till and each time it happens.

    Yeah man I completely forgot about it, It was just that he responds on each and every thread so that's why I was curious and got my answers too.

    Part of being CEO i guess. Doe a little of it all including marketing and support

    Oh yes, he does it all. That reminds me, has anyone here used the Pomodoro technique and found it efficient?

  • DeusVultDeusVult Member

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:
    My vps is basically idle at the moment.

    The only VPS you have?

    Yes

    Waiting for a good project to make good use of the VPS?

    Yeah. I just don't know what to code.
    My wayback machine project is fun, but if i would actually deploy it, it would require a lot of storage. And i primarily make it to learn more about web crawling and puppeteer.

    Oh that's cool. Yah, wayback machine would require a hell lot of storage.

    Well one thing that is different than the WA, is that i only save full-page screenshots instead of the whole page with all the assets.

    Oh so a waybackscreenshots. Sounds coolm.in that case you could compress the images using various tools. Infact, @TrK is working on a commercial image optimisation CDN that might be of help for your project

    Yeah average image size is currently around 25-100 kb. Small pages are around 25-50 kb, while large pages can easily reach 300 kb.

    You can bring them down a bit more.

    How? I am currently using the avif image format.

    I am not sure right now. But, I did work on this a few years ago. BTW, how are you compressing currently?

    First, the page screenshot is using png, then i compress to webp and then i compress that webp image to avif, but this only saves like 5-30 kb at the moment. Also the avif compression takes like 30 seconds.
    I am using the nodejs sharp library. I will send my code soon.

    That's quite impressive. Just to test though, try running the original image through tinypng.com and see if you are already able to get it a lower size than your final compressed image, if you haven't already. Just to confirm.

    Will try that. Don't have my laptop with me at the moment.

    Sure, let me know when you have. I am interested in this as well

    Oh by the way, the crawler already has added 800k urls in the db. About 5 of them are actually screenshotted (as that part is still work in progress).

    Wow!!!that's a lot bro

  • noob404noob404 Member

    @xpress7 said:

    @noob404 said:

    @xpress7 said:
    Lots of pages.. dont' want to go back now. Whats the topic currently? :D

    Img compression best methods. @BasToTheMax wants to know

    TrK knows better. He is building one optimization service.

    Yah pointed him to TrK, but he himself recommended imgproxy

  • noob404noob404 Member

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:
    My vps is basically idle at the moment.

    The only VPS you have?

    Yes

    Waiting for a good project to make good use of the VPS?

    Yeah. I just don't know what to code.
    My wayback machine project is fun, but if i would actually deploy it, it would require a lot of storage. And i primarily make it to learn more about web crawling and puppeteer.

    Oh that's cool. Yah, wayback machine would require a hell lot of storage.

    Well one thing that is different than the WA, is that i only save full-page screenshots instead of the whole page with all the assets.

    Oh so a waybackscreenshots. Sounds coolm.in that case you could compress the images using various tools. Infact, @TrK is working on a commercial image optimisation CDN that might be of help for your project

    Yeah average image size is currently around 25-100 kb. Small pages are around 25-50 kb, while large pages can easily reach 300 kb.

    You can bring them down a bit more.

    How? I am currently using the avif image format.

    I am not sure right now. But, I did work on this a few years ago. BTW, how are you compressing currently?

    First, the page screenshot is using png, then i compress to webp and then i compress that webp image to avif, but this only saves like 5-30 kb at the moment. Also the avif compression takes like 30 seconds.
    I am using the nodejs sharp library. I will send my code soon.

    That's quite impressive. Just to test though, try running the original image through tinypng.com and see if you are already able to get it a lower size than your final compressed image, if you haven't already. Just to confirm.

    Will try that. Don't have my laptop with me at the moment.

    Sure, let me know when you have. I am interested in this as well

    Oh by the way, the crawler already has added 800k urls in the db. About 5 of them are actually screenshotted (as that part is still work in progress).

    Oh I see. The crawler coded in Node as well?

Sign In or Register to comment.