Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


I wrote a shell script that turns spare VPS into a CDN node in one click
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

I wrote a shell script that turns spare VPS into a CDN node in one click

dedipromodedipromo Member
edited July 2020 in General

As title suggests. There are loads of scripts that install LAMP or LEMP stacks. There are loads of scripts that install VPNs. There are even loads of scripts that install mail servers. But automatically installing CDN seems pretty rare.

I have a bunch of spare VPS (I guess this applies to many LET users lol) located all over the world. Therefore, I decided to turn them into CDN nodes so that they can accelerate my websites. By sharing it, I hope that it can be helpful to other people.

Github link

https://github.com/Har-Kuun/OneClickCDN
Stars and forks are appreciated.

System requirement

  • Ubuntu 20.04 LTS 64 bit, or Debian 10 (experimental)
  • 1 IPv4
  • At least 512 MB for running the CDN instance
  • For the very first time, building the program from source requires about 1.5 GB RAM. You may add SWAP to your VPS for this step.
  • Root access or sudo user

How to use

wget https://raw.githubusercontent.com/Har-Kuun/OneClickCDN/master/OneClickCDN.sh && sudo bash OneClickCDN.sh

Features of this script

  • Build Traffic Server from source
  • Add websites
  • Manage SSL (provide certificates/key or automatically issue certificates with Let's Encrypt)
  • Manage CDN websites, purge cache, view stats, change IP...
  • Guided configuration

Why Traffic Server?

Many people favor Varnish or Nginx. Traffic Server is kind of old fashioned, but it still works great. It behaves decent in very high load. I tested a CDN node (1 E5 core, 1 GB RAM, 30 Mbps port) with loader.io using 2000 concurrent users sending 380000 requests continuously within 1 minute. The CDN node handled all of these successfully with average CPU load of only 40%. Check loader.io test result here https://bit.ly/32xD1nI

All suggestions, questions, and bug reports are highly appreciated. Thank you!

«13

Comments

  • JoeFJoeF Member
    edited July 2020

    What does the management interface look like? What statistics are available to view? Is it possible to centrally manage all the nodes from a master server?

  • @JoeF said:
    What does the management interface look like? What statistics are available to view? Is it possible to centrally manage all the nodes from a master server?

    From his project website (all in Chinese), I see the following for "interface"

    Thanked by 1dedipromo
  • jadenjaden Member

    I don't often click on shell scripts to run them on servers, but if you did have a desktop environment installed, wouldn't it take a double click to launch?

    Thanked by 1nocloud
  • dedipromodedipromo Member
    edited July 2020

    @daxterfellowes said:

    JoeF said:
    What does the management interface look like? What statistics are available to view? Is it possible to centrally manage all the nodes from a master server?

    From his project website (all in Chinese), I see the following for "interface"

    Yes, it's just bunch of options to allow you to choose from, not actually an interface.

    @JoeF said: What statistics are available to view?

    Here is a sample stats report.

                                   Traffic summary
    
    Origin Server                               Hits         Misses         Errors
    ------------------------------------------------------------------------------
    xx.xx.xx.xx:443                             86             80              0
    ==============================================================================
    
    
    
                                  162.244.95.22:443
    
    Request Result                         Count    Percent       Bytes    Percent
    ------------------------------------------------------------------------------
    Cache hit                                  1      0.60%    406.40KB     10.42%
    Cache hit RAM                              0      0.00%      0.00KB      0.00%
    Cache hit IMS                              0      0.00%      0.00KB      0.00%
    Cache hit refresh                         85     51.20%      1.12MB     29.46%
    Cache hit other                            0      0.00%      0.00KB      0.00%
    Cache hit total                           86     51.81%      1.52MB     39.88%
    
    Cache miss                                80     48.19%      2.29MB     60.12%
    Cache miss IMS                             0      0.00%      0.00KB      0.00%
    Cache miss refresh                         0      0.00%      0.00KB      0.00%
    Cache miss other                           0      0.00%      0.00KB      0.00%
    Cache miss total                          80     48.19%      2.29MB     60.12%
    
    Client aborted                             0      0.00%      0.00KB      0.00%
    Client read error                          0      0.00%      0.00KB      0.00%
    Connect failed                             0      0.00%      0.00KB      0.00%
    Invalid request                            0      0.00%      0.00KB      0.00%
    Unknown error(99)                          0      0.00%      0.00KB      0.00%
    Other errors                               0      0.00%      0.00KB      0.00%
    Errors total                               0      0.00%      0.00KB      0.00%
    ..............................................................................
    Total requests                           166    100.00%      3.81MB    100.00%
    
    
    HTTP return codes                      Count    Percent       Bytes    Percent
    ------------------------------------------------------------------------------
    100 Continue                               0      0.00%      0.00KB      0.00%
    200 OK                                   165     99.40%      3.81MB     99.98%
    201 Created                                0      0.00%      0.00KB      0.00%
    202 Accepted                               0      0.00%      0.00KB      0.00%
    203 Non-Authoritative Info                 0      0.00%      0.00KB      0.00%
    204 No content                             0      0.00%      0.00KB      0.00%
    205 Reset Content                          0      0.00%      0.00KB      0.00%
    206 Partial content                        0      0.00%      0.00KB      0.00%
    2xx Total                                165     99.40%      3.81MB     99.98%
    
    300 Multiple Choices                       0      0.00%      0.00KB      0.00%
    301 Moved permanently                      0      0.00%      0.00KB      0.00%
    302 Found                                  0      0.00%      0.00KB      0.00%
    303 See Other                              0      0.00%      0.00KB      0.00%
    304 Not modified                           0      0.00%      0.00KB      0.00%
    305 Use Proxy                              0      0.00%      0.00KB      0.00%
    307 Temporary Redirect                     0      0.00%      0.00KB      0.00%
    3xx Total                                  0      0.00%      0.00KB      0.00%
    
    400 Bad request                            0      0.00%      0.00KB      0.00%
    401 Unauthorized                           0      0.00%      0.00KB      0.00%
    402 Payment Required                       0      0.00%      0.00KB      0.00%
    403 Forbidden                              0      0.00%      0.00KB      0.00%
    404 Not found                              1      0.60%      0.73KB      0.02%
    405 Method Not Allowed                     0      0.00%      0.00KB      0.00%
    406 Not Acceptable                         0      0.00%      0.00KB      0.00%
    407 Proxy Auth Required                    0      0.00%      0.00KB      0.00%
    408 Request Timeout                        0      0.00%      0.00KB      0.00%
    409 Conflict                               0      0.00%      0.00KB      0.00%
    410 Gone                                   0      0.00%      0.00KB      0.00%
    411 Length Required                        0      0.00%      0.00KB      0.00%
    412 Precondition Failed                    0      0.00%      0.00KB      0.00%
    413 Request Entity Too Large               0      0.00%      0.00KB      0.00%
    414 Request-URI Too Long                   0      0.00%      0.00KB      0.00%
    415 Unsupported Media Type                 0      0.00%      0.00KB      0.00%
    416 Req Range Not Satisfiable              0      0.00%      0.00KB      0.00%
    417 Expectation Failed                     0      0.00%      0.00KB      0.00%
    4xx Total                                  1      0.60%      0.73KB      0.02%
    
    500 Internal Server Error                  0      0.00%      0.00KB      0.00%
    501 Not implemented                        0      0.00%      0.00KB      0.00%
    502 Bad gateway                            0      0.00%      0.00KB      0.00%
    503 Service unavailable                    0      0.00%      0.00KB      0.00%
    504 Gateway Timeout                        0      0.00%      0.00KB      0.00%
    505 HTTP Ver. Not Supported                0      0.00%      0.00KB      0.00%
    5xx Total                                  0      0.00%      0.00KB      0.00%
    
    000 Unknown                                0      0.00%      0.00KB      0.00%
    
    
    Origin hierarchies                     Count    Percent       Bytes    Percent
    ------------------------------------------------------------------------------
    NONE                                       1      0.60%    406.40KB     10.42%
    DIRECT                                   165     99.40%      3.41MB     89.58%
    SIBLING                                    0      0.00%      0.00KB      0.00%
    PARENT                                     0      0.00%      0.00KB      0.00%
    EMPTY                                      0      0.00%      0.00KB      0.00%
    invalid                                    0      0.00%      0.00KB      0.00%
    other                                      0      0.00%      0.00KB      0.00%
    
    
    HTTP Methods                           Count    Percent       Bytes    Percent
    ------------------------------------------------------------------------------
    OPTIONS                                    0      0.00%      0.00KB      0.00%
    GET                                      165     99.40%      3.81MB     99.98%
    HEAD                                       1      0.60%      0.59KB      0.02%
    POST                                       0      0.00%      0.00KB      0.00%
    PUT                                        0      0.00%      0.00KB      0.00%
    DELETE                                     0      0.00%      0.00KB      0.00%
    TRACE                                      0      0.00%      0.00KB      0.00%
    CONNECT                                    0      0.00%      0.00KB      0.00%
    PURGE                                      0      0.00%      0.00KB      0.00%
    none (-)                                   0      0.00%      0.00KB      0.00%
    other                                      0      0.00%      0.00KB      0.00%
    
    
    URL Schemes                            Count    Percent       Bytes    Percent
    ------------------------------------------------------------------------------
    HTTP (port 80)                             0      0.00%      0.00KB      0.00%
    HTTPS (port 443)                         166    100.00%      3.81MB    100.00%
    none                                       0      0.00%      0.00KB      0.00%
    other                                      0      0.00%      0.00KB      0.00%
    
    
    Protocols                              Count    Percent       Bytes    Percent
    ------------------------------------------------------------------------------
    IPv4                                     166    100.00%      3.81MB    100.00%
    IPv6                                       0      0.00%      0.00KB      0.00%
    
    
    Content Types                          Count    Percent       Bytes    Percent
    ------------------------------------------------------------------------------
    text/javascript                            0      0.00%      0.00KB      0.00%
    text/css                                   6      3.61%    942.32KB     24.15%
    text/html                                 71     42.77%    836.64KB     21.44%
    text/xml                                   0      0.00%      0.00KB      0.00%
    text/plain                                 0      0.00%      0.00KB      0.00%
    text/ other                                0      0.00%      0.00KB      0.00%
    text/ total                               77     46.39%      1.74MB     45.60%
    
    image/jpeg                                 4      2.41%      1.20MB     31.42%
    image/gif                                  0      0.00%      0.00KB      0.00%
    image/png                                  3      1.81%    145.90KB      3.74%
    image/bmp                                  0      0.00%      0.00KB      0.00%
    image/ other                               4      2.41%      5.22KB      0.13%
    image/ total                              11      6.63%      1.34MB     35.29%
    
    audio/x-wav                                0      0.00%      0.00KB      0.00%
    audio/x-mpeg                               0      0.00%      0.00KB      0.00%
    audio/ other                               0      0.00%      0.00KB      0.00%
    audio/ total                               0      0.00%      0.00KB      0.00%
    
    application/x-shockwave                    0      0.00%      0.00KB      0.00%
    application/[x-]javascript                73     43.98%    459.95KB     11.79%
    application/x-quicktime                    0      0.00%      0.00KB      0.00%
    application/zip                            0      0.00%      0.00KB      0.00%
    application/rss+xml                        0      0.00%      0.00KB      0.00%
    application/rss+atom                       0      0.00%      0.00KB      0.00%
    application/ other                         2      1.20%     58.37KB      1.50%
    application/ total                        75     45.18%    518.32KB     13.29%
    
    none                                       0      0.00%      0.00KB      0.00%
    other                                      3      1.81%    227.32KB      5.83%
    
    
    Elapsed time stats          Min          Max              Avg    Std Deviation
    ------------------------------------------------------------------------------
    Cache hit                   123          123           123.00             0.00
    Cache hit RAM                 0            0             0.00             0.00
    Cache hit IMS                 0            0             0.00             0.00
    Cache hit refresh             1          441            36.67            71.81
    Cache hit other               0            0             0.00             0.00
    Cache hit total               1          441            37.69            71.99
    Cache miss                    2          296            50.49            66.31
    Cache miss IMS                0            0             0.00             0.00
    Cache miss refresh            0            0             0.00             0.00
    Cache miss other              0            0             0.00             0.00
    Cache miss total              2          296            50.49            66.31
    
    ______________________________________________________________________________
    
    

    @JoeF said: Is it possible to centrally manage all the nodes from a master server?

    This is possible if you install Traffic Control http://traffic-control-cdn.readthedocs.io/en/latest/index.html but it is not included in the script itself.

  • @jaden said:
    I don't often click on shell scripts to run them on servers, but if you did have a desktop environment installed, wouldn't it take a double click to launch?

    You'll have to run this in shell with sudo users. Double click won't work I think.

  • umiumi Member

    Huge script from China that needs sudo privilegies. I'll pass

    Thanked by 2webcraft GreenPeace
  • WebProjectWebProject Host Rep, Veteran

    @umi said:
    Huge script from China that needs sudo privilegies. I'll pass

    Very small script and open source without any odd code, see yourself:
    https://github.com/Har-Kuun/OneClickCDN/blob/master/OneClickCDN.sh

    Based on your comments you can work for BBC TV.

  • PieHasBeenEatenPieHasBeenEaten Member, Host Rep

    @umi So I guess you hate Chinese people?

  • A quick scan of the script and it looks promising.

    Thanked by 2dedipromo tux
  • umiumi Member

    @PieHasBeenEaten said:
    @umi So I guess you hate Chinese people?

    I love Chinese people and don't like backdoors, especially when private keys are involved.

    Thanked by 1Master_Bo
  • I don’t see where you specify other servers to it into a cluster/network together. Am I missing something?

    Thanked by 1dedipromo
  • @doughnet said:
    I don’t see where you specify other servers to it into a cluster/network together. Am I missing something?

    Ah you'll have to run the script on each server that you'd like to use, then use DNS round robin/failover to point your domain to these IPs.

  • @dedipromo said:

    @doughnet said:
    I don’t see where you specify other servers to it into a cluster/network together. Am I missing something?

    Ah you'll have to run the script on each server that you'd like to use, then use DNS round robin/failover to point your domain to these IPs.

    Thanks. What about keeping content/data synced between all the servers?

    Thanked by 1dedipromo
  • OujiOuji Member

    @umi said:

    @PieHasBeenEaten said:
    @umi So I guess you hate Chinese people?

    I love Chinese people and don't like backdoors, especially when private keys are involved.

    There is no backdoor here though?

    Thanked by 1dedipromo
  • @doughnet said:

    @dedipromo said:

    @doughnet said:
    I don’t see where you specify other servers to it into a cluster/network together. Am I missing something?

    Ah you'll have to run the script on each server that you'd like to use, then use DNS round robin/failover to point your domain to these IPs.

    Thanks. What about keeping content/data synced between all the servers?

    These are CDN nodes, so they don't really store data. They only cache data from the origin server, so I think there is no need to sync data between all these servers.

  • @dedipromo said:

    @doughnet said:

    @dedipromo said:

    @doughnet said:
    I don’t see where you specify other servers to it into a cluster/network together. Am I missing something?

    Ah you'll have to run the script on each server that you'd like to use, then use DNS round robin/failover to point your domain to these IPs.

    Thanks. What about keeping content/data synced between all the servers?

    These are CDN nodes, so they don't really store data. They only cache data from the origin server, so I think there is no need to sync data between all these servers.

    Thanks. So caching the data could then in theory be 200gb if for example your site was hosting iso or videos?

  • Well done on the script.
    Good documentation on script in git will help on new comers and interested parties.

    Thanked by 2dedipromo tux
  • @doughnet said:

    @dedipromo said:

    @doughnet said:

    @dedipromo said:

    @doughnet said:
    I don’t see where you specify other servers to it into a cluster/network together. Am I missing something?

    Ah you'll have to run the script on each server that you'd like to use, then use DNS round robin/failover to point your domain to these IPs.

    Thanks. What about keeping content/data synced between all the servers?

    These are CDN nodes, so they don't really store data. They only cache data from the origin server, so I think there is no need to sync data between all these servers.

    Thanks. So caching the data could then in theory be 200gb if for example your site was hosting iso or videos?

    Yes, exactly.

  • @mrclown said:
    Well done on the script.
    Good documentation on script in git will help on new comers and interested parties.

    Thank you for the suggestion!

  • looks awesome, thanks!

    Thanked by 1dedipromo
  • Is this work like cloud flare where the cdn sit between the origin and automatically cache static resource, or some thing like bunnycdn?

  • @yokowasis said:
    Is this work like cloud flare where the cdn sit between the origin and automatically cache static resource, or some thing like bunnycdn?

    This is just installation script. ATS works like any other cache which is used by CF or bunny. Of cuz there'll be performance differences depending on what stack and how much optimization was made across their deployment.

    For simple setup, this script will help for beginners. In production setup, that needs to tune properly.

    Thanked by 2dedipromo vimalware
  • @yokowasis said:
    Is this work like cloud flare where the cdn sit between the origin and automatically cache static resource, or some thing like bunnycdn?

    Upon installation, this will work like cloudflare, where the CDN nodes sit between the origin and automatically cach static (and dynamic, if configured) resource.

  • wonder if there's a way to have it cache things in front of a Plex Server. I know people do this with CF but a lot of times end up getting banned. Probably be easy to do with Emby since it doesn't have to deal with the idiotic cloud authentication that Plex requires.

  • HarambeHarambe Member, Host Rep

    @doughnet said:
    wonder if there's a way to have it cache things in front of a Plex Server. I know people do this with CF but a lot of times end up getting banned. Probably be easy to do with Emby since it doesn't have to deal with the idiotic cloud authentication that Plex requires.

    There's no caching going on in those setups, they're just proxying traffic through CF to get around network issues.

    You could definitely setup your own proxy though, following basically the same steps as those CF guides - just adding in some iptables rules on the VM in the middle.

  • @doughnet said:
    wonder if there's a way to have it cache things in front of a Plex Server. I know people do this with CF but a lot of times end up getting banned. Probably be easy to do with Emby since it doesn't have to deal with the idiotic cloud authentication that Plex requires.

    I don't really know the answer and can only speculate. If the Plex Server respond with a Cache Control: no-cache header (which it likely will), this CDN will not cache it; if not, and the CDN is configured to cache dynamic content (which indeed is by default) then it should be able to cache the content.

  • @umi said:
    Huge script from China that needs sudo privilegies. I'll pass

    It's open source you can check it by yourself

    Thanked by 1dedipromo
  • I suggest an uninstall script.

    Also in the readme put a description of the commands. After installing I can't get back to the menu configuration list. There's no mention of ANY commands in the README.

    Thanked by 1dedipromo
  • @doughnet said:
    I suggest an uninstall script.

    Also in the readme put a description of the commands. After installing I can't get back to the menu configuration list. There's no mention of ANY commands in the README.

    Thanks for the feedback. I'll update it tomorrow. Actually, running the script again will bring you to the menu.

  • mrclownmrclown Member
    edited July 2020

    This could be another beginning like CENTMINMOD if maintained it well. But CDN doesn't work alone, need a good cluster to work together with fair amount of control and monitoring.

    This script is just a cache/reverse-proxy. Would be good to rename or extend to support bigger scale as a vision. Otherwise gonna mislead more people to compare with cloudflare, bunny and etc.

Sign In or Register to comment.