Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Quick Links


Shells Virtual Desktop
BMail.ag - Secure Email Service
Server.net
CPLicense.net
VPS Server
Buy VPN
Vultr
VMs for AI
HostDare
ReliableSite White-Label Dedicated Hosting for Resellers
InterServer VPS
BMail.ag - Secure Email Service
Best VPN
High-Performance Bare Metal Server Solutions
Karvl.com
Server Mania Cloud Hosting
DataWagon Hosting
AlphaVPS Hosting
Evoxt.com
Clouvider
VPS Hosting with NVMe
Residential IPs in the US & 4G Mobile Proxies in EU & US with Unlimited Bandwidth
ReliableSite White-Label Dedicated Hosting for Resellers
Rabisu - Hosting Solutions
Shells Virtual Desktop

Categories

In this Discussion

Home › Offers
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

SPANKING NEW DEALS! HAPPY NEW YEAR w/ PREMIUM GIVEAWAYS! FREEBIES GALORE by RackNerd.

1…471472474476477…1047

Comments

  • BasToTheMaxBasToTheMax Member

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:
    My vps is basically idle at the moment.

    The only VPS you have?

    Yes

    Waiting for a good project to make good use of the VPS?

    Yeah. I just don't know what to code.
    My wayback machine project is fun, but if i would actually deploy it, it would require a lot of storage. And i primarily make it to learn more about web crawling and puppeteer.

    Oh that's cool. Yah, wayback machine would require a hell lot of storage.

    Well one thing that is different than the WA, is that i only save full-page screenshots instead of the whole page with all the assets.

    Oh so a waybackscreenshots. Sounds coolm.in that case you could compress the images using various tools. Infact, @TrK is working on a commercial image optimisation CDN that might be of help for your project

    Yeah average image size is currently around 25-100 kb. Small pages are around 25-50 kb, while large pages can easily reach 300 kb.

    You can bring them down a bit more.

    How? I am currently using the avif image format.

    I am not sure right now. But, I did work on this a few years ago. BTW, how are you compressing currently?

    First, the page screenshot is using png, then i compress to webp and then i compress that webp image to avif, but this only saves like 5-30 kb at the moment. Also the avif compression takes like 30 seconds.
    I am using the nodejs sharp library. I will send my code soon.

    That's quite impressive. Just to test though, try running the original image through tinypng.com and see if you are already able to get it a lower size than your final compressed image, if you haven't already. Just to confirm.

    Will try that. Don't have my laptop with me at the moment.

    Sure, let me know when you have. I am interested in this as well

    Oh by the way, the crawler already has added 800k urls in the db. About 5 of them are actually screenshotted (as that part is still work in progress).

    Oh I see. The crawler coded in Node as well?

    Yes 😅

  • noob404noob404 Member

    @DeusVult said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:
    My vps is basically idle at the moment.

    The only VPS you have?

    Yes

    Waiting for a good project to make good use of the VPS?

    Yeah. I just don't know what to code.
    My wayback machine project is fun, but if i would actually deploy it, it would require a lot of storage. And i primarily make it to learn more about web crawling and puppeteer.

    Oh that's cool. Yah, wayback machine would require a hell lot of storage.

    Well one thing that is different than the WA, is that i only save full-page screenshots instead of the whole page with all the assets.

    Oh so a waybackscreenshots. Sounds coolm.in that case you could compress the images using various tools. Infact, @TrK is working on a commercial image optimisation CDN that might be of help for your project

    Yeah average image size is currently around 25-100 kb. Small pages are around 25-50 kb, while large pages can easily reach 300 kb.

    You can bring them down a bit more.

    How? I am currently using the avif image format.

    I am not sure right now. But, I did work on this a few years ago. BTW, how are you compressing currently?

    First, the page screenshot is using png, then i compress to webp and then i compress that webp image to avif, but this only saves like 5-30 kb at the moment. Also the avif compression takes like 30 seconds.
    I am using the nodejs sharp library. I will send my code soon.

    That's quite impressive. Just to test though, try running the original image through tinypng.com and see if you are already able to get it a lower size than your final compressed image, if you haven't already. Just to confirm.

    Will try that. Don't have my laptop with me at the moment.

    Sure, let me know when you have. I am interested in this as well

    Oh by the way, the crawler already has added 800k urls in the db. About 5 of them are actually screenshotted (as that part is still work in progress).

    Wow!!!that's a lot bro

    Wayback. That's expected.

  • noob404noob404 Member

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:
    My vps is basically idle at the moment.

    The only VPS you have?

    Yes

    Waiting for a good project to make good use of the VPS?

    Yeah. I just don't know what to code.
    My wayback machine project is fun, but if i would actually deploy it, it would require a lot of storage. And i primarily make it to learn more about web crawling and puppeteer.

    Oh that's cool. Yah, wayback machine would require a hell lot of storage.

    Well one thing that is different than the WA, is that i only save full-page screenshots instead of the whole page with all the assets.

    Oh so a waybackscreenshots. Sounds coolm.in that case you could compress the images using various tools. Infact, @TrK is working on a commercial image optimisation CDN that might be of help for your project

    Yeah average image size is currently around 25-100 kb. Small pages are around 25-50 kb, while large pages can easily reach 300 kb.

    You can bring them down a bit more.

    How? I am currently using the avif image format.

    I am not sure right now. But, I did work on this a few years ago. BTW, how are you compressing currently?

    First, the page screenshot is using png, then i compress to webp and then i compress that webp image to avif, but this only saves like 5-30 kb at the moment. Also the avif compression takes like 30 seconds.
    I am using the nodejs sharp library. I will send my code soon.

    That's quite impressive. Just to test though, try running the original image through tinypng.com and see if you are already able to get it a lower size than your final compressed image, if you haven't already. Just to confirm.

    Will try that. Don't have my laptop with me at the moment.

    Sure, let me know when you have. I am interested in this as well

    Oh by the way, the crawler already has added 800k urls in the db. About 5 of them are actually screenshotted (as that part is still work in progress).

    Oh I see. The crawler coded in Node as well?

    Yes 😅

    Oh so you are a Node head. I love PHP. Want to learn something new. Confused between Python and Node.

    Thanked by 1BasToTheMax
  • BeniskickbuttBeniskickbutt Member

    @noob404 said:

    @Beniskickbutt said:
    Oh wow you guys been busy today 🙂

    Covered some great topics. Since youa re here, let us choose a new topic, if you want

    I noticed! Any interesting stand outs?

  • noob404noob404 Member

    @Beniskickbutt said:

    @noob404 said:

    @Beniskickbutt said:
    Oh wow you guys been busy today 🙂

    Covered some great topics. Since youa re here, let us choose a new topic, if you want

    I noticed! Any interesting stand outs?

    Img compression seems to be a topic under discussion. Got any good recommendation.

  • BeniskickbuttBeniskickbutt Member

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:
    My vps is basically idle at the moment.

    The only VPS you have?

    Yes

    Waiting for a good project to make good use of the VPS?

    Yeah. I just don't know what to code.
    My wayback machine project is fun, but if i would actually deploy it, it would require a lot of storage. And i primarily make it to learn more about web crawling and puppeteer.

    Oh that's cool. Yah, wayback machine would require a hell lot of storage.

    Well one thing that is different than the WA, is that i only save full-page screenshots instead of the whole page with all the assets.

    Oh so a waybackscreenshots. Sounds coolm.in that case you could compress the images using various tools. Infact, @TrK is working on a commercial image optimisation CDN that might be of help for your project

    Yeah average image size is currently around 25-100 kb. Small pages are around 25-50 kb, while large pages can easily reach 300 kb.

    You can bring them down a bit more.

    How? I am currently using the avif image format.

    I am not sure right now. But, I did work on this a few years ago. BTW, how are you compressing currently?

    First, the page screenshot is using png, then i compress to webp and then i compress that webp image to avif, but this only saves like 5-30 kb at the moment. Also the avif compression takes like 30 seconds.
    I am using the nodejs sharp library. I will send my code soon.

    That's quite impressive. Just to test though, try running the original image through tinypng.com and see if you are already able to get it a lower size than your final compressed image, if you haven't already. Just to confirm.

    Will try that. Don't have my laptop with me at the moment.

    Another site to add to the pool of bookmarks. Haven't heard of that one

  • BeniskickbuttBeniskickbutt Member

    @noob404 said:

    @Beniskickbutt said:

    @noob404 said:

    @TrK said:

    @noob404 said:

    @TrK said:
    another random suggestion: Don't compile mariadb 11.4 on a slow VPS.... it will take hours to finish the task. or just compile it and go back to sleep... it will be done when you wake up... probably....

    Remember trying to compile PHP. Afew years ago on a 2GB VPS. Made me wait for 2 hours and then said you need atleast 4GB RAM! :lol:

    well different cases for different tasks you can bypass the ram requirements with swap but i don't usually wait more than an hour for my compilation tasks.

    Oh yes. Infact there was a workaround (non-swap) for building it that I learned of back then. Don't remember now. I prefer my personal PC for compilation.

    That's what I usually try to do as well. Local compile remote deploy

    My preferred method as well. Somehow I feel I got more control when it's local.

    My local box is also a little better than the remotes which is why i feel better doing it that way

  • noob404noob404 Member

    @Beniskickbutt said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:
    My vps is basically idle at the moment.

    The only VPS you have?

    Yes

    Waiting for a good project to make good use of the VPS?

    Yeah. I just don't know what to code.
    My wayback machine project is fun, but if i would actually deploy it, it would require a lot of storage. And i primarily make it to learn more about web crawling and puppeteer.

    Oh that's cool. Yah, wayback machine would require a hell lot of storage.

    Well one thing that is different than the WA, is that i only save full-page screenshots instead of the whole page with all the assets.

    Oh so a waybackscreenshots. Sounds coolm.in that case you could compress the images using various tools. Infact, @TrK is working on a commercial image optimisation CDN that might be of help for your project

    Yeah average image size is currently around 25-100 kb. Small pages are around 25-50 kb, while large pages can easily reach 300 kb.

    You can bring them down a bit more.

    How? I am currently using the avif image format.

    I am not sure right now. But, I did work on this a few years ago. BTW, how are you compressing currently?

    First, the page screenshot is using png, then i compress to webp and then i compress that webp image to avif, but this only saves like 5-30 kb at the moment. Also the avif compression takes like 30 seconds.
    I am using the nodejs sharp library. I will send my code soon.

    That's quite impressive. Just to test though, try running the original image through tinypng.com and see if you are already able to get it a lower size than your final compressed image, if you haven't already. Just to confirm.

    Will try that. Don't have my laptop with me at the moment.

    Another site to add to the pool of bookmarks. Haven't heard of that one

    Been using tinypng for a really long time. They good

  • BeniskickbuttBeniskickbutt Member

    @xpress7 said:
    Lots of pages.. dont' want to go back now. Whats the topic currently? :D

    looks like some chats about CDN and image optimizations

  • BeniskickbuttBeniskickbutt Member

    @DeusVult said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    You will probably win a prize imo. Everyone here will. But the value of the prize depends on your own work

    yep, i think most likely everyone htat is active at this very moment have a prize headed their way by the end :)

  • BeniskickbuttBeniskickbutt Member

    @DeusVult said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    Sure, dude. Stay active and you won't regret it.

    Yeah, lets see btw why is dustinc not visible didn't see him yesterday and even today too

    I guess he's on holidays

    even CEOs gotta rest i suppose ;) I thought it was all work an no play with those folks

  • noob404noob404 Member

    @Beniskickbutt said:

    @noob404 said:

    @Beniskickbutt said:

    @noob404 said:

    @TrK said:

    @noob404 said:

    @TrK said:
    another random suggestion: Don't compile mariadb 11.4 on a slow VPS.... it will take hours to finish the task. or just compile it and go back to sleep... it will be done when you wake up... probably....

    Remember trying to compile PHP. Afew years ago on a 2GB VPS. Made me wait for 2 hours and then said you need atleast 4GB RAM! :lol:

    well different cases for different tasks you can bypass the ram requirements with swap but i don't usually wait more than an hour for my compilation tasks.

    Oh yes. Infact there was a workaround (non-swap) for building it that I learned of back then. Don't remember now. I prefer my personal PC for compilation.

    That's what I usually try to do as well. Local compile remote deploy

    My preferred method as well. Somehow I feel I got more control when it's local.

    My local box is also a little better than the remotes which is why i feel better doing it that way

    I was offered a 128GB VPS for compiling Batocera. But being uncomfortable I decided to do it with my ancient i5-5250u :lol:

  • BeniskickbuttBeniskickbutt Member

    @noob404 said:

    @Beniskickbutt said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    Sure, dude. Stay active and you won't regret it.

    Yeah, lets see btw why is dustinc not visible didn't see him yesterday and even today too

    Let's not forget the he's the CEO of RackNerd and that it's still the holiday season, being one of the busiest times for them. So, he might be extremely busy. But rest assured. He will be here with many GAs. Hope you can hang around till and each time it happens.

    Yeah I'm surprised he even has as much time as he does to drop in here

    Yah, he has some great time management skills

    im still interested in knowing how he keeps up double bandwidth gang. Is it a cool script that helps him or does he really manually comb through it all??

  • BeniskickbuttBeniskickbutt Member

    @noob404 said:
    All the talk of local compilation just reminded me, I once compiled Batocera from source on my i5-5250u. Took 5 days!

    thats obsurd! Any reason why you compiled rather than just got a built distribution of it? Are you hacking on linux kernels now too?

  • cainyxuescainyxues Member

    @Beniskickbutt said:

    @noob404 said:

    @Beniskickbutt said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    Sure, dude. Stay active and you won't regret it.

    Yeah, lets see btw why is dustinc not visible didn't see him yesterday and even today too

    Let's not forget the he's the CEO of RackNerd and that it's still the holiday season, being one of the busiest times for them. So, he might be extremely busy. But rest assured. He will be here with many GAs. Hope you can hang around till and each time it happens.

    Yeah I'm surprised he even has as much time as he does to drop in here

    Yah, he has some great time management skills

    im still interested in knowing how he keeps up double bandwidth gang. Is it a cool script that helps him or does he really manually comb through it all??

    I think there might be someone from his team too only viewing threads or maybe regex script

  • BeniskickbuttBeniskickbutt Member

    @cainyxues said:

    @DeusVult said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    You will probably win a prize imo. Everyone here will. But the value of the prize depends on your own work

    Lets see and just hope for the best for everyone

    no need to doubt your self! We are pretty sure you'll get something here by the end ez

    Thanked by 1cainyxues
  • noob404noob404 Member

    @Beniskickbutt said:

    @noob404 said:

    @Beniskickbutt said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    Sure, dude. Stay active and you won't regret it.

    Yeah, lets see btw why is dustinc not visible didn't see him yesterday and even today too

    Let's not forget the he's the CEO of RackNerd and that it's still the holiday season, being one of the busiest times for them. So, he might be extremely busy. But rest assured. He will be here with many GAs. Hope you can hang around till and each time it happens.

    Yeah I'm surprised he even has as much time as he does to drop in here

    Yah, he has some great time management skills

    im still interested in knowing how he keeps up double bandwidth gang. Is it a cool script that helps him or does he really manually comb through it all??

    I'd like to think that he does it using a script or addon. But, I have a feeling he does it manually.

  • BeniskickbuttBeniskickbutt Member

    @DeusVult said:

    @Beniskickbutt said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    Sure, dude. Stay active and you won't regret it.

    Yeah, lets see btw why is dustinc not visible didn't see him yesterday and even today too

    Let's not forget the he's the CEO of RackNerd and that it's still the holiday season, being one of the busiest times for them. So, he might be extremely busy. But rest assured. He will be here with many GAs. Hope you can hang around till and each time it happens.

    Yeah I'm surprised he even has as much time as he does to drop in here

    Same. Initially I thought it was his team, but it's him.

    hes on another level but that type of time management and effiency i guess is just a requirement to become a ceo

  • noob404noob404 Member

    @Beniskickbutt said:

    @noob404 said:
    All the talk of local compilation just reminded me, I once compiled Batocera from source on my i5-5250u. Took 5 days!

    thats obsurd! Any reason why you compiled rather than just got a built distribution of it? Are you hacking on linux kernels now too?

    Was compiling it for the PS4, so, it did require many modifications.

  • BeniskickbuttBeniskickbutt Member

    @noob404 said:

    @Beniskickbutt said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @TrK said:
    another random suggestion: Don't compile mariadb 11.4 on a slow VPS.... it will take hours to finish the task. or just compile it and go back to sleep... it will be done when you wake up... probably....

    Remember trying to compile PHP. Afew years ago on a 2GB VPS. Made me wait for 2 hours and then said you need atleast 4GB RAM! :lol:

    lol :lol:

    Oh that was a disaster. To overcome this, I tried compiling on my PC and moving it to the VPS. only after I installed it did I know that the specific module for which I rebuilt PHP won't work without a few more dependencies. Learned akt that day

    akt means?

    Pardon I am on my phone. I meant "a lot"

    Your English is pretty good. You mentioned wanting to get better. A lot of people use alot instead

    We have been taught to use both depending on the sentence

    interesting.. alot is actually not a word :)

    Thanked by 1noob404
  • noob404noob404 Member

    @Beniskickbutt said:

    @DeusVult said:

    @Beniskickbutt said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    Sure, dude. Stay active and you won't regret it.

    Yeah, lets see btw why is dustinc not visible didn't see him yesterday and even today too

    Let's not forget the he's the CEO of RackNerd and that it's still the holiday season, being one of the busiest times for them. So, he might be extremely busy. But rest assured. He will be here with many GAs. Hope you can hang around till and each time it happens.

    Yeah I'm surprised he even has as much time as he does to drop in here

    Same. Initially I thought it was his team, but it's him.

    hes on another level but that type of time management and effiency i guess is just a requirement to become a ceo

    Oh yes. It's a tough field as well.

  • BeniskickbuttBeniskickbutt Member

    @xpress7 said:

    @noob404 said:

    @xpress7 said:
    Lots of pages.. dont' want to go back now. Whats the topic currently? :D

    Img compression best methods. @BasToTheMax wants to know

    TrK knows better. He is building one optimization service.

    going to open source it as well i think i saw ;) Thats very considerate. I would love to be more active in the open source community but sometimes its hard to let go of your babies

  • noob404noob404 Member

    @Beniskickbutt said:

    @noob404 said:

    @Beniskickbutt said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @TrK said:
    another random suggestion: Don't compile mariadb 11.4 on a slow VPS.... it will take hours to finish the task. or just compile it and go back to sleep... it will be done when you wake up... probably....

    Remember trying to compile PHP. Afew years ago on a 2GB VPS. Made me wait for 2 hours and then said you need atleast 4GB RAM! :lol:

    lol :lol:

    Oh that was a disaster. To overcome this, I tried compiling on my PC and moving it to the VPS. only after I installed it did I know that the specific module for which I rebuilt PHP won't work without a few more dependencies. Learned akt that day

    akt means?

    Pardon I am on my phone. I meant "a lot"

    Your English is pretty good. You mentioned wanting to get better. A lot of people use alot instead

    We have been taught to use both depending on the sentence

    interesting.. alot is actually not a word :)

    Didn't know this. Got to learn a lot about grammar and vocabulary from you

  • BeniskickbuttBeniskickbutt Member

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:
    My vps is basically idle at the moment.

    The only VPS you have?

    Yes

    Waiting for a good project to make good use of the VPS?

    Yeah. I just don't know what to code.
    My wayback machine project is fun, but if i would actually deploy it, it would require a lot of storage. And i primarily make it to learn more about web crawling and puppeteer.

    Oh that's cool. Yah, wayback machine would require a hell lot of storage.

    Well one thing that is different than the WA, is that i only save full-page screenshots instead of the whole page with all the assets.

    Oh so a waybackscreenshots. Sounds coolm.in that case you could compress the images using various tools. Infact, @TrK is working on a commercial image optimisation CDN that might be of help for your project

    Yeah average image size is currently around 25-100 kb. Small pages are around 25-50 kb, while large pages can easily reach 300 kb.

    You can bring them down a bit more.

    How? I am currently using the avif image format.

    I am not sure right now. But, I did work on this a few years ago. BTW, how are you compressing currently?

    First, the page screenshot is using png, then i compress to webp and then i compress that webp image to avif, but this only saves like 5-30 kb at the moment. Also the avif compression takes like 30 seconds.
    I am using the nodejs sharp library. I will send my code soon.

    That's quite impressive. Just to test though, try running the original image through tinypng.com and see if you are already able to get it a lower size than your final compressed image, if you haven't already. Just to confirm.

    Will try that. Don't have my laptop with me at the moment.

    Sure, let me know when you have. I am interested in this as well

    Oh by the way, the crawler already has added 800k urls in the db. About 5 of them are actually screenshotted (as that part is still work in progress).

    wait a minute.. are you crwaling way back machine? Just looked back in the quote chain.. thats a bum load of data

  • cainyxuescainyxues Member

    @noob404 said:

    @Beniskickbutt said:

    @noob404 said:
    All the talk of local compilation just reminded me, I once compiled Batocera from source on my i5-5250u. Took 5 days!

    thats obsurd! Any reason why you compiled rather than just got a built distribution of it? Are you hacking on linux kernels now too?

    Was compiling it for the PS4, so, it did require many modifications.

    WTH what are you guys here I thought you were just talking about little bit of programming but you all are taking it to the next level :lol:

  • BeniskickbuttBeniskickbutt Member

    @noob404 said:

    @Beniskickbutt said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    Sure, dude. Stay active and you won't regret it.

    Yeah, lets see btw why is dustinc not visible didn't see him yesterday and even today too

    Let's not forget the he's the CEO of RackNerd and that it's still the holiday season, being one of the busiest times for them. So, he might be extremely busy. But rest assured. He will be here with many GAs. Hope you can hang around till and each time it happens.

    Yeah man I completely forgot about it, It was just that he responds on each and every thread so that's why I was curious and got my answers too.

    Part of being CEO i guess. Doe a little of it all including marketing and support

    Oh yes, he does it all. That reminds me, has anyone here used the Pomodoro technique and found it efficient?

    That just means also adding breaks right? I use a timer to make sure i get up on periodic intervals. Its good for your health

  • noob404noob404 Member

    Okay will join back in a few hours maybe. You guys enjoy the party.

  • BeniskickbuttBeniskickbutt Member

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:
    My vps is basically idle at the moment.

    The only VPS you have?

    Yes

    Waiting for a good project to make good use of the VPS?

    Yeah. I just don't know what to code.
    My wayback machine project is fun, but if i would actually deploy it, it would require a lot of storage. And i primarily make it to learn more about web crawling and puppeteer.

    Oh that's cool. Yah, wayback machine would require a hell lot of storage.

    Well one thing that is different than the WA, is that i only save full-page screenshots instead of the whole page with all the assets.

    Oh so a waybackscreenshots. Sounds coolm.in that case you could compress the images using various tools. Infact, @TrK is working on a commercial image optimisation CDN that might be of help for your project

    Yeah average image size is currently around 25-100 kb. Small pages are around 25-50 kb, while large pages can easily reach 300 kb.

    You can bring them down a bit more.

    How? I am currently using the avif image format.

    I am not sure right now. But, I did work on this a few years ago. BTW, how are you compressing currently?

    First, the page screenshot is using png, then i compress to webp and then i compress that webp image to avif, but this only saves like 5-30 kb at the moment. Also the avif compression takes like 30 seconds.
    I am using the nodejs sharp library. I will send my code soon.

    That's quite impressive. Just to test though, try running the original image through tinypng.com and see if you are already able to get it a lower size than your final compressed image, if you haven't already. Just to confirm.

    Will try that. Don't have my laptop with me at the moment.

    Sure, let me know when you have. I am interested in this as well

    Oh by the way, the crawler already has added 800k urls in the db. About 5 of them are actually screenshotted (as that part is still work in progress).

    Oh I see. The crawler coded in Node as well?

    He should try rust :)

  • BeniskickbuttBeniskickbutt Member

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:

    @noob404 said:

    @BasToTheMax said:
    My vps is basically idle at the moment.

    The only VPS you have?

    Yes

    Waiting for a good project to make good use of the VPS?

    Yeah. I just don't know what to code.
    My wayback machine project is fun, but if i would actually deploy it, it would require a lot of storage. And i primarily make it to learn more about web crawling and puppeteer.

    Oh that's cool. Yah, wayback machine would require a hell lot of storage.

    Well one thing that is different than the WA, is that i only save full-page screenshots instead of the whole page with all the assets.

    Oh so a waybackscreenshots. Sounds coolm.in that case you could compress the images using various tools. Infact, @TrK is working on a commercial image optimisation CDN that might be of help for your project

    Yeah average image size is currently around 25-100 kb. Small pages are around 25-50 kb, while large pages can easily reach 300 kb.

    You can bring them down a bit more.

    How? I am currently using the avif image format.

    I am not sure right now. But, I did work on this a few years ago. BTW, how are you compressing currently?

    First, the page screenshot is using png, then i compress to webp and then i compress that webp image to avif, but this only saves like 5-30 kb at the moment. Also the avif compression takes like 30 seconds.
    I am using the nodejs sharp library. I will send my code soon.

    That's quite impressive. Just to test though, try running the original image through tinypng.com and see if you are already able to get it a lower size than your final compressed image, if you haven't already. Just to confirm.

    Will try that. Don't have my laptop with me at the moment.

    Sure, let me know when you have. I am interested in this as well

    Oh by the way, the crawler already has added 800k urls in the db. About 5 of them are actually screenshotted (as that part is still work in progress).

    Oh I see. The crawler coded in Node as well?

    Yes 😅

    Oh so you are a Node head. I love PHP. Want to learn something new. Confused between Python and Node.

    I would say python seems like a "better" thing for this but really all languages are capable of doing almots anything now with the amount of libraries they have

  • DeusVultDeusVult Member

    @noob404 said:

    @Beniskickbutt said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:

    @noob404 said:

    @cainyxues said:
    I too want credits man like It would fund vps's if for instance even 100usd :lol: it would be good for 10 years :smile:

    You can easily convert your win's price value from trh main prize pool to RackNerd credits

    Yeah, I know that it's just that first of all I need to win :lol: lets just hope for the best for now................................................. :smile: :smiley:

    Sure, dude. Stay active and you won't regret it.

    Yeah, lets see btw why is dustinc not visible didn't see him yesterday and even today too

    Let's not forget the he's the CEO of RackNerd and that it's still the holiday season, being one of the busiest times for them. So, he might be extremely busy. But rest assured. He will be here with many GAs. Hope you can hang around till and each time it happens.

    Yeah I'm surprised he even has as much time as he does to drop in here

    Yah, he has some great time management skills

    I want that as well lol

Sign In or Register to comment.