All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Made a looking-glass clone for fun
Hi LET,
As a weekend project, I decided to give Deno another try and tried to build a looking glass with it. I'm not unhappy with the existing PHP one, just thought it would be a good project to try out a new tool. The goal is to have most of the visible functionalities as the PHP one, and I was really curious to see if it's possible to pack everything inside a single executable.
It kinda worked, but the way Deno "compiles" things is that it just bundles your code and then bundle that bundle with the Deno executable, which is fine, but the file size is not optimial, around 80MB...
It generates the config file on the first run if it doesn't exists, and warns about missing commands. The config file should be self-explanatory.
Demo: http://199.195.253.86/
Code: https://github.com/ericls/looking-glass
Download: https://github.com/ericls/looking-glass/releases
Feel free to take a look, fork it, play with it or don't care about it
Comments
It's the first Deno project I read, and it's surprising easy to understand (revision
5347815918cd1ac341d3b65b73aec72e9456cbb6
).I'm concerned on the performance of downloadable files.
Downloadable files on a looking glass is intended to be representative of network bandwidth.
Is
crypto.getRandomValues(p)
fast enough so that it would not become a bottleneck?I see some attempt on preventing ping and traceroute to private addresses.
However, I can easily get around this by publishing an A record that point to a private address.
RateLimiter#clientCounter
is only added to but never deleted from.It would consume too much memory after many different IPs have accessed the socket, especially when the server is deployed on IPv6 endpoint.
You should delete a counter when it reaches zero, and not storing zero in the object.
Thanks for you reply!
Good observations.
I had some of the same thoughts, except the one with private IP bypass, which is a good find.
Not sure about the performance of
getRandomValues
, but I read it on MDN that it should be fine. I don't have enough data to support it tho.I think tho, that it's always insecure to run user submitted things on a non-sandboxed environment, not sure how to go about that...
I havent read the code but from my understanding secure random usually involve reading from environment, devices without enough entropy might slow it down.
Again, sorry that I havent read the code. In Node.js (which I believe is very similar to Deno), you can use VM to run them in separate context. But anything calling shell command is tricky to restrict, make sure you use whitelist approach instead of blacklist.
Ok finally decided to go in front my laptop and read the code. I would say the code quality is decent Can be better but quite good.
Some comments:
Try not to use
window.
object to store global data but instead use a Singleton, it will make TypeScript hinting more accurate and easier to read.If the setting includes single quote (eg. inside
siteTitle
), the following code will break:https://github.com/ericls/looking-glass/blob/3bb51bdeadf562e323b36911e6b441ad3ec5fe63/src/index.tsx#L81-L90
The solution that we usually use is double encoding, not optimal for size but more secure. Also try to replace the character
<
into\u003c
after encoding. Basically:(Same for classes)
Your rate limiter only record for success request (eg.
JSON.parse
being valid), recommend moving it to the top to also record failed request.isValidHost
is not very safe but Deno seems to auto encode all the arguments before passing to shell so should be relatively safe.Might also want to consider adding a timeout in the Deno code instead of relying on Shell to return. (It can be as easy as
setTimeout
or usePromise.race()
)As mentioned by yoursunny, generating static files might be faster than randomly generate on the fly. You can generate it one-off when looking glass is first initialised/executed.
Ok feel like I am doing a code review now sorry - Anyway code looks clean, keep it up!
I did a quick benchmark.
Hardware: Xeon Gold 6240, 100Gbps connection.
Server runs in Docker container.
ericls/looking-glass v0.2.0:
Caddy file server, NVMe drive, non-first load:
Entropy is not a concern.
Deno
crypto.getRandomValues
is implemented with Rustthread_rng.fill()
.Rust RNG is implemented with
getrandom
syscall.getrandom(2) draws entropy from the urandom source that does not block for gathering entropy.
Wouldn't it be fastest to serve the test files from a ramdisk?
So this has one of the small issues I had with DNSTools a while back: The ping tries to block pinging private networks (eg. 192.168.x.x or 10.x.x.x), but it doesn't block it if you use a hostname that resolves to a private IP (eg. try
la03.int.d.sb
).You may also want to consider showing timeouts (no reply) via
-O
. The exact command I use on DNSTools isping -i 0.5 -c 5 -O <ip>
: https://github.com/Daniel15/dnstools/blob/master/src/DnsTools.Worker/Tools/Ping.cs#L41Why are you using SIGKILL (
process.kill(9)
) rather than SIGTERM when reading stderr? https://github.com/ericls/looking-glass/blob/3bb51bdeadf562e323b36911e6b441ad3ec5fe63/src/socket.ts#L64I like this idea, basically what we actually need is just some of the bytes to be random enough. When the app first started, we can initialise 50% of available memory in the heap. After that, store some random generated bytes in the heap memory and serve them whenever there's request to test files. (In case the requested file is larger than available memory, just do a modulo)
Everytime I see SIGKILL /
-9
it reminds me of this comicaww poor SIGKILL'd penguins (...are those even penguins?)
Demo seems to be offline, if you'd like a free DDoS protected VPS to host it on, shoot me a PM
With kernel buffer cache, I used to expect only minor difference, except on first load.
A quick benchmark indicates that serving from a ramdisk increases download speed by 13% on second load.
Caddy file server, NVMe drive, first load and second load.
Before starting the HTTP server and testing first load, I executed
sync; echo 3 > /proc/sys/vm/drop_caches
to flush the buffer cache.Caddy file server,
/dev/shm
mount, first load and second load.I flushed the buffer cache using the same procedure, but noticed that
free -m
command indicates more than 100GB still in the buffer cache, suggesting that files in/dev/shm
persist in the buffer cache.@FAT32 Thanks so much for reviewing
@yoursunny Thanks for benchmarking it!
I probably don't have time during the week to fix the issues but should be able to fix them on the weekend.
There's one really pretty bad bug on oak that crashes the server if there's no host header...
https://github.com/oakserver/oak/issues/386
Hopefully that can be solved as well.
For entropy (if any issues) use haveged. Such simple application often overlooked.
Didn't see your comments before for some reason. Wow, great feedback!
@FAT32 @yoursunny
Got some time to update it.
https://github.com/ericls/looking-glass/releases/tag/v0.3.0
You should put indication of action. At the moment, if my ISP do not allow PING and I perform ping test, it just shows nothing and suddenly spits out 100% loss result.
Neat script, thought, a bit overcomplicated with Deno.
Good point. It's not printing stderror right now...