Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


CDNPerf v2 is live - All feedback is appreciated
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

CDNPerf v2 is live - All feedback is appreciated

We rebuilt CDNPerf and launched it a while ago.

  • Its 100% static HTML, no dynamic parts.
  • We now use our own data pulled from PerfOps
  • Fixed many bugs
  • Improved UX

All serious feedback is appreciated. Let me know how we can improve it, what to fix what features and tools to add!

Thanks

Comments

  • cheers for this!

    Thanked by 1jimaek
  • I really dig your work @jimaek. Nice to see BunnyCDN on the list. :)

    Thanked by 2jimaek luissousa
  • NeoonNeoon Community Contributor, Veteran

    Premium.

  • It seems all RUM UPTIME are not good.

  • niknik Member, Host Rep

    Apart from the the fact that this is a great tool, the problem I see with this is that latency is not really important. The available bandwidth is important if there is no bandwidth available to a specific ISP it doesn't matter if the latency is 10ms or 50ms.

    Let's say I am choosing a CDN based on the fact that I want to livestream to an audience of 10k people. 2k of those have DTAG as ISP and the stream has a bitrate of 5mbps. That means the CDN needs to be able to handle 10 Gbps DTAG traffic, it doesn't matter if the livestreaming files are delivered in 10ms or 50ms, but it matters if the bandwidth to DTAG is available at all, otherwise the whole livestream is not watchable.

    I like the idea of BunnyCDN but I doubt that they are competitive in that regard. Last time I checked they also don't operate their own network, so they have zero influence on that.

    Thanked by 2PUSHR_Victor chrisp
  • OnApp_TerryOnApp_Terry Member
    edited April 2018

    @nik said:
    Apart from the the fact that this is a great tool, the problem I see with this is that latency is not really important. The available bandwidth is important if there is no bandwidth available to a specific ISP it doesn't matter if the latency is 10ms or 50ms.

    Let's say I am choosing a CDN based on the fact that I want to livestream to an audience of 10k people. 2k of those have DTAG as ISP and the stream has a bitrate of 5mbps. That means the CDN needs to be able to handle 10 Gbps DTAG traffic, it doesn't matter if the livestreaming files are delivered in 10ms or 50ms, but it matters if the bandwidth to DTAG is available at all, otherwise the whole livestream is not watchable.

    I like the idea of BunnyCDN but I doubt that they are competitive in that regard. Last time I checked they also don't operate their own network, so they have zero influence on that.

    There is a difference between testing performance and capacity. For the majority of visitors to the site, performance will be more important than capacity. Those that have capacity concerns -- or edge use cases, will typically request a consultation/PoC up front. They know they're in a unique situation.

    ... and really, your example use case is one that is fairly easily solved. @BunnyCDN could source the necessary capacity in a few hours. Is work required? Absolutely. But when you're a smaller company like BunnyCDN, CDN.net, and others, that level of work is required to win these type of deals. It's going that extra mile to care for the customer that allows smaller providers to win deals like that.

  • niknik Member, Host Rep
    edited April 2018

    @OnApp_Terry said:

    @nik said:
    Apart from the the fact that this is a great tool, the problem I see with this is that latency is not really important. The available bandwidth is important if there is no bandwidth available to a specific ISP it doesn't matter if the latency is 10ms or 50ms.

    Let's say I am choosing a CDN based on the fact that I want to livestream to an audience of 10k people. 2k of those have DTAG as ISP and the stream has a bitrate of 5mbps. That means the CDN needs to be able to handle 10 Gbps DTAG traffic, it doesn't matter if the livestreaming files are delivered in 10ms or 50ms, but it matters if the bandwidth to DTAG is available at all, otherwise the whole livestream is not watchable.

    I like the idea of BunnyCDN but I doubt that they are competitive in that regard. Last time I checked they also don't operate their own network, so they have zero influence on that.

    There is a difference between testing performance and capacity. For the majority of visitors to the site, performance will be more important than capacity.

    ... and really, your example use case is one that is fairly easily solved. @BunnyCDN could source the necessary capacity in a few hours. Is work required? Absolutely. But when you're a smaller company like BunnyCDN, CDN.net, and others, that level of work is required to win these type of deals. It's going that extra mile to care for the customer that allows smaller providers to win deals like that.

    So you are telling me that if Netflix or YouTube is stuttering every other second that is absolutely fine and that I have a better latency to those services is more important anyways?

    Where do you source 10 Gbps of DTAG capacity within a few hours for less than $10/TB?

  • @nik Well its not that easy to do that. We are focusing on HTTP performance and latency which is already an expensive thing to do but has a wider audience and makes more sense to compare CDN providers with.

  • niknik Member, Host Rep

    @jimaek said:
    @nik Well its not that easy to do that. We are focusing on HTTP performance and latency which is already an expensive thing to do but has a wider audience and makes more sense to compare CDN providers with.

    Of course, I know that this would be a hard nut to crack, but your ranking implies that CDN.net and BunnyCDN are nearly as good as Akamai which is very misleading. But I understand that we will probably never see a ranking that is measuring "real world performance".

  • OnApp_TerryOnApp_Terry Member
    edited April 2018

    @nik said:

    @OnApp_Terry said:

    @nik said:
    Apart from the the fact that this is a great tool, the problem I see with this is that latency is not really important. The available bandwidth is important if there is no bandwidth available to a specific ISP it doesn't matter if the latency is 10ms or 50ms.

    Let's say I am choosing a CDN based on the fact that I want to livestream to an audience of 10k people. 2k of those have DTAG as ISP and the stream has a bitrate of 5mbps. That means the CDN needs to be able to handle 10 Gbps DTAG traffic, it doesn't matter if the livestreaming files are delivered in 10ms or 50ms, but it matters if the bandwidth to DTAG is available at all, otherwise the whole livestream is not watchable.

    I like the idea of BunnyCDN but I doubt that they are competitive in that regard. Last time I checked they also don't operate their own network, so they have zero influence on that.

    There is a difference between testing performance and capacity. For the majority of visitors to the site, performance will be more important than capacity.

    ... and really, your example use case is one that is fairly easily solved. @BunnyCDN could source the necessary capacity in a few hours. Is work required? Absolutely. But when you're a smaller company like BunnyCDN, CDN.net, and others, that level of work is required to win these type of deals. It's going that extra mile to care for the customer that allows smaller providers to win deals like that.

    So you are telling me that if Netflix or YouTube is stuttering every other second that is absolutely fine and that I have a better latency to those services is more important anyways?

    Where do you source 10 Gbps of DTAG within a few hours for less than $10/TB?

    The average use case isn't Netflix or YouTube. Catering the website to an edge case is pointless. I don't think that's the market CDNperf is designed towards. It's made to help the 80% of websites out there that at best might push a few terabytes per month.

    If I have a prospect with those requirements, I'm first going to see if they're okay broadcasting to multiple PoPs from the origin. Then I'm going to contact providers that I know have peering at AMS-IX, LINX and DE-CIX. In Europe, on a decent commit I'm getting bandwidth for $2.50 a terabyte.

  • PUSHR_VictorPUSHR_Victor Member, Host Rep
    edited April 2018

    Guess the only misleading thing here is that CDNperf does not state they are not measuring throughput at all and only the shortest ping time to the closest PoP. No issue with that, just needs a bit of a clarification so potential customers know what they are looking at.

    PS. Excluding the 500byte image.

    Thanked by 1OnApp_Terry
  • niknik Member, Host Rep

    @OnApp_Terry said:

    @nik said:

    @OnApp_Terry said:

    @nik said:
    Apart from the the fact that this is a great tool, the problem I see with this is that latency is not really important. The available bandwidth is important if there is no bandwidth available to a specific ISP it doesn't matter if the latency is 10ms or 50ms.

    Let's say I am choosing a CDN based on the fact that I want to livestream to an audience of 10k people. 2k of those have DTAG as ISP and the stream has a bitrate of 5mbps. That means the CDN needs to be able to handle 10 Gbps DTAG traffic, it doesn't matter if the livestreaming files are delivered in 10ms or 50ms, but it matters if the bandwidth to DTAG is available at all, otherwise the whole livestream is not watchable.

    I like the idea of BunnyCDN but I doubt that they are competitive in that regard. Last time I checked they also don't operate their own network, so they have zero influence on that.

    There is a difference between testing performance and capacity. For the majority of visitors to the site, performance will be more important than capacity.

    ... and really, your example use case is one that is fairly easily solved. @BunnyCDN could source the necessary capacity in a few hours. Is work required? Absolutely. But when you're a smaller company like BunnyCDN, CDN.net, and others, that level of work is required to win these type of deals. It's going that extra mile to care for the customer that allows smaller providers to win deals like that.

    So you are telling me that if Netflix or YouTube is stuttering every other second that is absolutely fine and that I have a better latency to those services is more important anyways?

    Where do you source 10 Gbps of DTAG within a few hours for less than $10/TB?

    The average use case isn't Netflix or YouTube. Catering a website to edge case is pointless. And I don't think that's the market CDNperf is designed towards. It's made to help the 80% of websites out there that at best might push a few terabytes per month.

    If I have a prospect with those requirements, I'm first going to see if they're okay broadcasting to multiple PoPs from the origin. Then I'm going to contact providers that I know have peering at AMS-IX, LINX and DE-CIX. In Europe, on a decent commit I'm getting bandwidth for $2.50 a terabyte.

    Well, I disagree, delivering live video is not an edge case. Netflix and YouTube were examples, there are thousands of other sites that produce huge amounts of traffic with (live) video. You even advertise the following on CDN.net:

    "Serve multi-format video without stutters and buffers, to create your own slick ‘tube’ experience."

    How do you make sure that it's not stuttering or buffering? It's exactly the example I pointed out.

    DE-CIX won't help you here in Germany. DTAG has around 40% market share and the only thing they might do via DE-CIX is selling you very very expensive transit so a provider with a DE-CIX uplink does nothing to help here.

  • BunnySpeedBunnySpeed Member, Host Rep
    edited April 2018

    @nik said:

    @jimaek said:
    @nik Well its not that easy to do that. We are focusing on HTTP performance and latency which is already an expensive thing to do but has a wider audience and makes more sense to compare CDN providers with.

    Of course, I know that this would be a hard nut to crack, but your ranking implies that CDN.net and BunnyCDN are nearly as good as Akamai which is very misleading. But I understand that we will probably never see a ranking that is measuring "real world performance".

    How is it misleading? It's actually the fact that we don't run our own network that allows us to have such a good performance with this level of pricing. We do actually have DTAG, DE-CIX and many other peers in Frankfurt, because we use carefully selected providers that already have the capacity in place. We keep plenty of free capacity as well, but we don't allow live streaming anyway. A CDN is not just all network either, we put a lot of effort into optimal routing, which also one of the reason reason why BunnyCDN performs well.

    I've started noticing this a lot that some people assume BunnyCDN is some kind of subpar product, even comparing it to just a bunch of servers, while other CDNs are magic, just because it's cheap. Actually it's cheap because we don't have to rent out a 10Gbit DTAG line just to use DTAG, but we already have access to it due to our hosting partners. I made a ton of effort using millions of credits on ATLAS Ripe to make sure things are routed optimally and we listened to user feedback to slowly build a really well connected network that we want to offer for a decent price.

    We also make heavy use of automation, no phone support etc and relatively low advertisement which also results in a significant saving.

    Plus, the big players have completely custom pricing for larger users anyway that's even lower from what we can offer.

    Thanked by 2Jord vimalware
  • @nik said:

    @OnApp_Terry said:

    @nik said:

    @OnApp_Terry said:

    @nik said:
    Apart from the the fact that this is a great tool, the problem I see with this is that latency is not really important. The available bandwidth is important if there is no bandwidth available to a specific ISP it doesn't matter if the latency is 10ms or 50ms.

    Let's say I am choosing a CDN based on the fact that I want to livestream to an audience of 10k people. 2k of those have DTAG as ISP and the stream has a bitrate of 5mbps. That means the CDN needs to be able to handle 10 Gbps DTAG traffic, it doesn't matter if the livestreaming files are delivered in 10ms or 50ms, but it matters if the bandwidth to DTAG is available at all, otherwise the whole livestream is not watchable.

    I like the idea of BunnyCDN but I doubt that they are competitive in that regard. Last time I checked they also don't operate their own network, so they have zero influence on that.

    There is a difference between testing performance and capacity. For the majority of visitors to the site, performance will be more important than capacity.

    ... and really, your example use case is one that is fairly easily solved. @BunnyCDN could source the necessary capacity in a few hours. Is work required? Absolutely. But when you're a smaller company like BunnyCDN, CDN.net, and others, that level of work is required to win these type of deals. It's going that extra mile to care for the customer that allows smaller providers to win deals like that.

    So you are telling me that if Netflix or YouTube is stuttering every other second that is absolutely fine and that I have a better latency to those services is more important anyways?

    Where do you source 10 Gbps of DTAG within a few hours for less than $10/TB?

    The average use case isn't Netflix or YouTube. Catering a website to edge case is pointless. And I don't think that's the market CDNperf is designed towards. It's made to help the 80% of websites out there that at best might push a few terabytes per month.

    If I have a prospect with those requirements, I'm first going to see if they're okay broadcasting to multiple PoPs from the origin. Then I'm going to contact providers that I know have peering at AMS-IX, LINX and DE-CIX. In Europe, on a decent commit I'm getting bandwidth for $2.50 a terabyte.

    Well, I disagree, delivering live video is not an edge case. Netflix and YouTube were examples, there are thousands of other sites that produce huge amounts of traffic with (live) video. You even advertise the following on CDN.net:

    "Serve multi-format video without stutters and buffers, to create your own slick ‘tube’ experience."

    How do you make sure that it's not stuttering or buffering? It's exactly the example I pointed out.

    DE-CIX won't help you here in Germany. DTAG has around 40% market share and the only thing they might do via DE-CIX is selling you very very expensive transit so a provider with a DE-CIX uplink does nothing to help here.

    ... and we can do it the majority of the time. BunnyCDN does it as well. Seriously, go on Reddit and look who provides the CDN for some of the image hosting sites. @BunnyCDN and I share stories on Discord about how we can tell a goal is scored, because /r/soccer is suddenly pushing a few gbits to a previously quiet resource.

    Anyway, DTAG peers at DE-CIX. I wouldn't contact DTAG directly -- I'd go with someone that has an existing relationship.

  • filefile Member

    It's rare that one has to think in terms of Google, Netflix, or Youtube realistically. As much as we'd like to think so - we're not that big and don't have the same constraints. In fact thinking in those terms can end up limiting you/slowing you down. (This applies not just to the above but also tools and other things).

  • PUSHR_VictorPUSHR_Victor Member, Host Rep
    edited April 2018

    @nik said:

    @OnApp_Terry said:

    @nik said:

    @OnApp_Terry said:

    @nik said:
    Apart from the the fact that this is a great tool, the problem I see with this is that latency is not really important. The available bandwidth is important if there is no bandwidth available to a specific ISP it doesn't matter if the latency is 10ms or 50ms.

    Let's say I am choosing a CDN based on the fact that I want to livestream to an audience of 10k people. 2k of those have DTAG as ISP and the stream has a bitrate of 5mbps. That means the CDN needs to be able to handle 10 Gbps DTAG traffic, it doesn't matter if the livestreaming files are delivered in 10ms or 50ms, but it matters if the bandwidth to DTAG is available at all, otherwise the whole livestream is not watchable.

    I like the idea of BunnyCDN but I doubt that they are competitive in that regard. Last time I checked they also don't operate their own network, so they have zero influence on that.

    There is a difference between testing performance and capacity. For the majority of visitors to the site, performance will be more important than capacity.

    ... and really, your example use case is one that is fairly easily solved. @BunnyCDN could source the necessary capacity in a few hours. Is work required? Absolutely. But when you're a smaller company like BunnyCDN, CDN.net, and others, that level of work is required to win these type of deals. It's going that extra mile to care for the customer that allows smaller providers to win deals like that.

    So you are telling me that if Netflix or YouTube is stuttering every other second that is absolutely fine and that I have a better latency to those services is more important anyways?

    Where do you source 10 Gbps of DTAG within a few hours for less than $10/TB?

    The average use case isn't Netflix or YouTube. Catering a website to edge case is pointless. And I don't think that's the market CDNperf is designed towards. It's made to help the 80% of websites out there that at best might push a few terabytes per month.

    If I have a prospect with those requirements, I'm first going to see if they're okay broadcasting to multiple PoPs from the origin. Then I'm going to contact providers that I know have peering at AMS-IX, LINX and DE-CIX. In Europe, on a decent commit I'm getting bandwidth for $2.50 a terabyte.

    Well, I disagree, delivering live video is not an edge case. Netflix and YouTube were examples, there are thousands of other sites that produce huge amounts of traffic with (live) video. You even advertise the following on CDN.net:

    "Serve multi-format video without stutters and buffers, to create your own slick ‘tube’ experience."

    How do you make sure that it's not stuttering or buffering? It's exactly the example I pointed out.

    DE-CIX won't help you here in Germany. DTAG has around 40% market share and the only thing they might do via DE-CIX is selling you very very expensive transit so a provider with a DE-CIX uplink does nothing to help here.

    Nah , wait, Live video and VOD are totally different things. Live is totally an edge case (unless ingested for remux/transcoding somehwere before the edge - then I agree). NetFlix and YT are VOD, not live. On a side note, willing to take this to PM if you can offer a dedi with DTAG for a reasonable price:)

Sign In or Register to comment.