Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Shells Virtual Desktop
BMail.ag - Secure Email Service
Server.net
CPLicense.net
VPS Server
Buy VPN
Vultr
VMs for AI
HostDare
ReliableSite White-Label Dedicated Hosting for Resellers
InterServer VPS
BMail.ag - Secure Email Service
Best VPN
High-Performance Bare Metal Server Solutions
Karvl.com
Server Mania Cloud Hosting
DataWagon Hosting
AlphaVPS Hosting
Evoxt.com
Clouvider
VPS Hosting with NVMe
Residential IPs in the US & 4G Mobile Proxies in EU & US with Unlimited Bandwidth
ReliableSite White-Label Dedicated Hosting for Resellers
Rabisu - Hosting Solutions
Shells Virtual Desktop
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

HAZI.ro | Performance drops expected tomorrow for VPSs in Romania

1101113151632

Comments

  • @Moopah said: Voxility is expensive

    He could just get an OVH 10Gbps box (or leverage existing) announce all IPs and be up in hours, but that's 50ms overhead?

    Really needs to get DDoS protection provider and GRE it back. Poor man solution but he would get back online.

    I wouldn't mind 50ms overhead if it meant my service was online.

    Who really goes to mid Romania for the best latency?

    Also outbound latency would still be direct out via RCS / Orange @FlorinMarian - Please do something like this.

  • AndreixAndreix Member, Host Rep

    @drivex said:

    @Kris said:

    @FlorinMarian said: As I said, I tried but that community is ignored.

    Didn't see that, thought you just put a ticket it. It would be very advantageous for them to open that up to you.

    Other note: seems like you know much about bgp. Any advice for good learning material?

    Fot start I can recommend CBTNuggets, if you can find materials.

    Thanked by 2drivex Kris
  • KrisKris Member
    edited January 2024

    @drivex said: Other note: seems like you know much about bgp. Any advice for good learning material?

    I learned it by poking around https://bgp.he.net/ looking at MTRs and finally worked at HostVirtual (now NetActuate) almost a decade ago, helping them become one of the largest peered ISPs in the world today.

    If I had the choice at the time, likely would've chosen one of these courses.

    At the time they were veering from low end VPS market and going into Anycast, which got me interested in BGP. One IP to anywhere was an awesome theory, and duplicating 1.1.1.1 or 8.8.8.8 (or 4.2.2.4 if you're old enough) was something I wanted to do.

    While HostVirtual's main thing was under 10ms to the anywhere, it took a LOT of peering. Using IXReach to get tiny data center locations to numerous nearby IX's. Sometimes you didn't have everyone on route-servers which meant an email to establish a relationship. I created an email template that got around a 75% conversion on turning up peering. This meant we bypassed transit - giving customers not only free bandwidth, but better routing.

    Finally, DDoS mitigation was a service where if someone tried to take you out, traffic would be diverted to a scrubbing centre.

    I took the very long route of learning it and wouldn't suggest it, honestly Fran at BuyVM offers a service for very cheap now where you can test and setup Anycast.

    Also got it implemented while working at Vultr, as it was $$$ at the time, and I felt HV was gatekeeping. Setting up a few BuyVM instances or Vultr + quagga would get you some knowledge and a number of those Udemy courses are great.

    I know a lot about communities and prepending, because when building a subnet / ASN that announces to the world, you didn't want sinkholes where Australia would route to LAX, even if you had a PoP in Australia and NZ, so quickly learned prepending is slap-dash and communities are great to engineer traffic. They can keep traffic in a region, not announced to international providers, etc.

    Sorry for the long post, but true BGP and Anycast enthusiast and I don't understand how 50ms overhead for a zero-cost solution is better than keeping the DC essentially offline.

    https://i.ping.pe/n/P/img_nPBTQbD4.png

  • FlorinMarianFlorinMarian Member, Host Rep

    @Kris said:

    @drivex said: Other note: seems like you know much about bgp. Any advice for good learning material?

    I learned it by poking around https://bgp.he.net/ looking at MTRs and finally worked at HostVirtual (now NetActuate) almost a decade ago, helping them become one of the largest peered ISPs in the world today.

    If I had the choice at the time, likely would've chosen one of these courses.

    At the time they were veering from low end VPS market and going into Anycast, which got me interested in BGP. One IP to anywhere was an awesome theory, and duplicating 1.1.1.1 or 8.8.8.8 (or 4.2.2.4 if you're old enough) was something I wanted to do.

    While HostVirtual's main thing was under 10ms to the anywhere, it took a LOT of peering. Using IXReach to get tiny data center locations to numerous nearby IX's. Sometimes you didn't have everyone on route-servers which meant an email to establish a relationship. I created an email template that got around a 75% conversion on turning up peering. This meant we bypassed transit - giving customers not only free bandwidth, but better routing.

    Finally, DDoS mitigation was a service where if someone tried to take you out, traffic would be diverted to a scrubbing centre.

    I took the very long route of learning it and wouldn't suggest it, honestly Fran at BuyVM offers a service for very cheap now where you can test and setup Anycast.

    Also got it implemented while working at Vultr, as it was $$$ at the time, and I felt HV was gatekeeping. Setting up a few BuyVM instances or Vultr + quagga would get you some knowledge and a number of those Udemy courses are great.

    I know a lot about communities and prepending, because when building a subnet / ASN that announces to the world, you didn't want sinkholes where Australia would route to LAX, even if you had a PoP in Australia and NZ, so quickly learned prepending is slap-dash and communities are great to engineer traffic. They can keep traffic in a region, not announced to international providers, etc.

    Sorry for the long post, but true BGP and Anycast enthusiast and I don't understand how 50ms overhead for a zero-cost solution is better than keeping the DC essentially offline.

    https://i.ping.pe/n/P/img_nPBTQbD4.png

    Waiting for OVH BYOIP setup for 2nd subnet. This may take up to 3 weeks.

  • KrisKris Member
    edited January 2024

    @FlorinMarian said: Waiting for OVH BYOIP setup for 2nd subnet. This may take up to 3 weeks.

    I'd go back and mention DDoS providers like Path and ones @dbContext mentined can LOA and have you up in hours, but do not want to go in circles.

    Maybe do some bog standard things like asking Orange to setup an edge firewall for you, add every tor exit node and.... well I've run out of ideas. Get fastnetmon setup (it takes minutes) with whatever flowspect / netflow you have and manually contact the NOC with IPs to null-route, if you aren't getting blanket attacked.

    Sorry if I've come off caustic in this thread, but it's really because I don't to see some skid with something as basic as TorsHammer take you offline.

  • FlorinMarianFlorinMarian Member, Host Rep
    cisco01.hazi.ro#show interfaces TenGigabitEthernet1/50 | include rate
      Queueing strategy: fifo
      5 minute input rate 4415255000 bits/sec, 421796 packets/sec
      5 minute output rate 170199000 bits/sec, 16563 packets/sec
    cisco01.hazi.ro#show interfaces GigabitEthernet1/1 | include rate
      Queueing strategy: fifo
      5 minute input rate 189452000 bits/sec, 23898 packets/sec
      5 minute output rate 292000 bits/sec, 61 packets/sec
    cisco01.hazi.ro#
    

    It is very strange what is happening.
    On the ports to which I am connected to the ISP, I see a lot of traffic and packets, and the dedicated servers do not receive high traffic at all (below 100Mbps). At the same time, from the servers to the switch, there are many timeouts and pings of the order of hundreds of ms.

  • MoopahMoopah Member
    edited January 2024

    fastnetmon (the SaaS service) looks suspiciously cheap for 10Gbit clean traffic

    Never mind, it's actually just licensing for the software product.

    Thanked by 1Kris
  • @FlorinMarian said:

    cisco01.hazi.ro#show interfaces TenGigabitEthernet1/50 | include rate
      Queueing strategy: fifo
      5 minute input rate 4415255000 bits/sec, 421796 packets/sec
      5 minute output rate 170199000 bits/sec, 16563 packets/sec
    cisco01.hazi.ro#show interfaces GigabitEthernet1/1 | include rate
      Queueing strategy: fifo
      5 minute input rate 189452000 bits/sec, 23898 packets/sec
      5 minute output rate 292000 bits/sec, 61 packets/sec
    cisco01.hazi.ro#
    

    It is very strange what is happening.
    On the ports to which I am connected to the ISP, I see a lot of traffic and packets, and the dedicated servers do not receive high traffic at all (below 100Mbps). At the same time, from the servers to the switch, there are many timeouts and pings of the order of hundreds of ms.

    And what conclusions you are drawing from this? Document this and it to the bachelor thesis.

  • edited January 2024

    @FlorinMarian said:

    cisco01.hazi.ro#show interfaces TenGigabitEthernet1/50 | include rate
      Queueing strategy: fifo
      5 minute input rate 4415255000 bits/sec, 421796 packets/sec
      5 minute output rate 170199000 bits/sec, 16563 packets/sec
    cisco01.hazi.ro#show interfaces GigabitEthernet1/1 | include rate
      Queueing strategy: fifo
      5 minute input rate 189452000 bits/sec, 23898 packets/sec
      5 minute output rate 292000 bits/sec, 61 packets/sec
    cisco01.hazi.ro#
    

    It is very strange what is happening.
    On the ports to which I am connected to the ISP, I see a lot of traffic and packets, and the dedicated servers do not receive high traffic at all (below 100Mbps). At the same time, from the servers to the switch, there are many timeouts and pings of the order of hundreds of ms.

    Depending on what kind of packets are coming in that might not be all that surprising. I could imagine like for example reflected RSTs/SYN-ACKs not making it past the switch while at the same time maximally exhausting it by coming as lots and lots of just about the tiniest packets possible.

  • <= 97.00% - The customer can request a 30% discount

    I guess there will be a lot of discounts for February, @FlorinMarian ?

    Thanked by 1yoursunny
  • @Moopah said: Never mind, it's actually just licensing for the software product.

    There's a community edition that can be installed in 10 minutes on a VM. Point netflow / sflow to that server. Boom, you will have grafana statistics.

    Idk why he's not given it a shot.

    https://github.com/pavel-odintsov/fastnetmon

    https://fastnetmon.com/install/

    Thanked by 1host_c
  • @Kris said:
    Idk why he's not given it a shot.

    You are like a math teacher trying to teach a housewife what PI is.
    Get some sleep instead. Do you think he will thank you, hire you or send you money for your help?

    He is in too deep with all his investments. Best to sell it all and colocate

    Thanked by 3Kris yoursunny adly
  • @packetnext said: Get some sleep instead. Do you think he will thank you, hire you or send you money for your help?

    No, I honestly just have been in the hosting field since 2000 (was HostDime's first staff member, worked with manny to make the site readable, all the way to building out a maitland aircraft hangar into DimeNOC, which I coined the name for.)

    Owned my own hosting company until the EIG types came by with unbeatable (albeit BS) disk space on plans, and sold my company to go to college full-time.

    High school with HostDime were fun times, but now I'm a graybeard angry old man in my mid 30s who's trying to help Romanian closet hosts who have no desire to do anything.

    I guess I was just hoping he was a response or two away from realizing the fastest solution is free and literally has a bash script to install. He seemed to be going in the right direction.

    Not here for any cash, help, and I have enough anxiety to want to work for him. I care that subnets are down for days and no plan is in site. Prob some neurological thing, but I like things being fixed. :(

    Thanked by 2Lex FlorinMarian
  • @packetnext said: Get some sleep instead.

    You're ultimately right.

    Thanked by 2host_c fluffernutter
  • yoursunnyyoursunny Member, IPv6 Advocate

    @Levi said: For some reason I feel that entity responsible for this incident is reading this thread and lol'ing outrageously.

    @host_c

  • LexLex Member

    Only a few weeks ago I replied to Florin that he shouldn't be asking the price he's asking for the services he's providing. One of the reasons being that other providers are not only better, but actually cheaper by up to 80%. Don't get me wrong, I'm not cheap. I'll gladly pay prem, but for prem quality.

    Seeing this thread and how he doesn't care about the customers he currently has, stating that he can always start over ... is just heart breaking for me honestly.

    BTW if nothing else about this company was a big red flag for you, this should definitely be.

    He states that he doesn't care, that's it's cheaper to start over. He's arrogant and blatantly refuses or ignores possibly good suggestions. Hell, he received offers for help for FREE from knowledgeable people in the industry (something he doesn't deserve imho) and he just DGAF

    After all this, his main concern is about latency?! Like fully timing out or a fully unreliable network is waaay better than +50ms!? Yeah, because Romania is the network hub of Europe with excelent ping with the whole world, LOL.

    He insists to talk with his provider, even though they send him to walk the dog (Romanian expression for go kick rocks) instead of going the faster routes (like other said, getting set up in minutes...)

    My brain just can't handle the logic of this human being.

    @FlorinMarian said: Also, after solving the situation, it's cheaper to lose all my clients who don't understand that this type of situation cannot be prevented with the budgets that both they and we have, than to charge my monthly bill with another 700$.

    Honestly, his business has/had a bad business plan from the ground up. Building any type of data center/data room has an insane ROI and you just can't compete with others. Not within the budget he had/has. (Maybe I'm wrong though)

    All of this could've been avoided greatly and he could've been way richer now if he would've just bought hardware, bought IPs and negotiated some sweet colocation deals. I know at least 2 providers within LEB budget that offer 10G dedicated for colocation. DDoS protection included. Or hell, even renting rack space probably would've had a better ROI than whatever he did. (Maybe I'm wrong though)

    The only thing that has some logic in my mind is if he somehow got funded and just burned through the cash, like those handouts for startups or something from the government/EU. (Maybe I'm wrong though)

  • @FlorinMarian said:

    @Kris said:

    @FlorinMarian said: Considering that there are barriers at the firmware level and I have the guaranteed bandwidth, I don't know how your scenario can happen.

    When you become a burden that it would take thousands just to cater to one account, they are unable to serve you anymore, and would simply sever ties, letting the lawyers deal with you.

    I seriously hope it doesn't happen, but I don't know of many ISPs that would tolerate what's going on without some effort by the end-user / enough knowledge to mitigate, or send communities upstream.

    If you're getting attacked on a single IP, tag it 8953:666 - it's the same at many ISPs.

    Get fastnetmon installed. With flowspec or Netflow (they actually were kind enough to get netflow lite integrated) you can get an eye inside your network so you aren't guessing who is getting attacked.

    It doesn't need to be setup to do RTBH, rather can give you an eye inside your network.

    I tried to blackhole an IP of mine from OVH, but the respective community is ignored.

    That’s because it’s the wrong way around, you block the destination not the source.

  • AndreixAndreix Member, Host Rep
  • FlorinMarianFlorinMarian Member, Host Rep

    @Lex said:
    Only a few weeks ago I replied to Florin that he shouldn't be asking the price he's asking for the services he's providing. One of the reasons being that other providers are not only better, but actually cheaper by up to 80%. Don't get me wrong, I'm not cheap. I'll gladly pay prem, but for prem quality.

    The price has nothing to do with the current situation. If I sold cheaper, I would only have more affected customers at this moment, nothing more.

    Seeing this thread and how he doesn't care about the customers he currently has, stating that he can always start over ... is just heart breaking for me honestly.

    • The sound of a sad violin is heard in the background
      Come on seriously, how can you accuse someone of carelessness when he's active here and respond to relevant messages and do everything he can according to his own strength and knowledge all day long?

    BTW if nothing else about this company was a big red flag for you, this should definitely be.

    It is. In all my other threads I talked about advanced DDoS protections from the most famous providers and suddenly it turned out that I lied.

    He states that he doesn't care, that's it's cheaper to start over. He's arrogant and blatantly refuses or ignores possibly good suggestions. Hell, he received offers for help for FREE from knowledgeable people in the industry (something he doesn't deserve imho) and he just DGAF

    Brother, when you are a taxi driver and at some point a wheel breaks on the car with the customer in the car and the passers-by tell you "Call a helicopter!" or "Call a plane to take him to his destination!" you're just stupid if you do it because that's what people say.
    You try to take your client to the destination, but not without taking into account how much the client pays you and how much you have to pay in turn to do this.

    After all this, his main concern is about latency?! Like fully timing out or a fully unreliable network is waaay better than +50ms!? Yeah, because Romania is the network hub of Europe with excelent ping with the whole world, LOL.

    You missed the part with BYOIP whose installation takes 3 weeks, more or less on purpose.

    He insists to talk with his provider, even though they send him to walk the dog (Romanian expression for go kick rocks) instead of going the faster routes (like other said, getting set up in minutes...)

    The earlier plane is definitely the fastest. So the client pays you $7 and the plane costs
    $70000.

    My brain just can't handle the logic of this human being.

    @FlorinMarian said: Also, after solving the situation, it's cheaper to lose all my clients who don't understand that this type of situation cannot be prevented with the budgets that both they and we have, than to charge my monthly bill with another 700$.

    Honestly, his business has/had a bad business plan from the ground up. Building any type of data center/data room has an insane ROI and you just can't compete with others. Not within the budget he had/has. (Maybe I'm wrong though)

    Now the lack of DDoS protection is felt, but the price per TB/month and per KW/h is still at prices far below the market price. You talk about competition as if I had made a master plan with 100 racks available in the location, but what can you see, I only filled one.

    All of this could've been avoided greatly and he could've been way richer now if he would've just bought hardware, bought IPs and negotiated some sweet colocation deals. I know at least 2 providers within LEB budget that offer 10G dedicated for colocation. DDoS protection included. Or hell, even renting rack space probably would've had a better ROI than whatever he did. (Maybe I'm wrong though)

    There is no such thing in Romania. If you don't believe me, counter me with an offer for 10U, 1200KWh/month and 2Gbps symmetrical guaranteed and unlimited under 750 EUR.
    Only Calin and his stories have cheap and very very very good colocation.

    The only thing that has some logic in my mind is if he somehow got funded and just burned through the cash, like those handouts for startups or something from the government/EU. (Maybe I'm wrong though)

    >
    Cool conspiracy but it doesn't make sense.
    All investments are from own funds out of passion and desire for knowledge, the rest are stories.

  • LET needs a down vote button, not a flag button, a down vote button for irritating posts

  • Should Florin be stripped of his Provider Tag? Or atleast make a tag that says Basement Provider.

  • Coming up next: Florin does a virmache style LET quit and fly unchallenged under admin protection in LES

    Thanked by 2emgh ScreenReader
  • @jmaxwell said:
    Coming up next: Florin does a virmache style LET quit and fly unchallenged under admin protection in LES

    don't you worry about that.

  • @Wicked said:
    <= 97.00% - The customer can request a 30% discount

    I guess there will be a lot of discounts for February, @FlorinMarian ?

    “The monthly value of the SLA index (Service Level Agreement = service availability) will be the value reported for a maximum uptime of at least 3 (three) of the following independent monitoring services: (…)”

    and

    “The absence of the report of at least 3 (three) of the monitoring services listed above concludes a 100% availability.”

  • host_chost_c Patron Provider, Top Host, Megathread Squad
    edited January 2024

    @Levi said: For some reason I feel that entity responsible for this incident is reading this thread and lol'ing outrageously.

    @host_c

    @yoursunny

    This is not funny, I was one of the first that contacted Florin to give him a hand, don't be a dick.

    Even that this is a competition among providers for sales, when one gets DDOS-ed, others step in to help ( just read thru the post's )

    Your network being fuked up is something that we do not wish for others to happen. I would rather battle in prices, products, services than to see any provider fail/go under because of this. That is a fair battle/competition, not shit that makes you network non operational, and makes client services unusable, that is trash in my opinion.

    But Bro has to understand, that it is time for him to take he's setup to the next level, and train himself to be prepared for Disaster Scenarios.

    PS: Florian, pleas do not post any more info with detail. Like for now we all know Your Uplink Ports name, one is for RDS and 1 is for Orange, they are called TenGigabitEthernet1/50 and GigabitEthernet1/1, we also know the make and model of your device. If you post the version also, then the one doing this can search for version vulnerabilities. Stop feeding sensitive info, just edit out the port names, and equipment name next time.

    Offer is still on the table, just like @c1vhosting and others, just say if we can help, and with what.

    Thanked by 1totally_not_banned
  • @tentor said:
    Why do you use LET and not sending email to the directly impacted customers?

    he is trying to impress !

    Thanked by 1Andreix
  • FlorinMarianFlorinMarian Member, Host Rep

    This morning I discovered an amazing thing.
    In addition to the DDoS attack, Orange also has something badly configured at the network level because:

    • with both ISPs active, Orange has 4.4 of 2Gbps used, always having over 400K packets/sec that never reach the servers but only on the GigabitEthernet port of the switch
    • with Orange deactivated and the Digi session of only 150Mbps, the infrastructure is standing, the traffic being about 4-40Mbps, 15K packets/sec.

    I'm waiting for their NOC to respond to the incident, but it's phenomenal how easily they send you to buy something more every time you ask them to investigate something.

    Evidence:

    Orange switchport off:
    ! RCS&RDS ISP
    cisco01.hazi.ro#show interfaces GigabitEthernet1/1 | include rate
      Queueing strategy: fifo
      5 minute input rate 40913000 bits/sec, 15117 packets/sec
      5 minute output rate 35129000 bits/sec, 4529 packets/sec
    
    
    Both ISPs:
    cisco01.hazi.ro#show interfaces TenGigabitEthernet1/50 | include rate
      Queueing strategy: fifo
      5 minute input rate 4283238000 bits/sec, 407071 packets/sec
      5 minute output rate 218782000 bits/sec, 21082 packets/sec
    cisco01.hazi.ro#! RCS&RDS ISP
    cisco01.hazi.ro#show interfaces GigabitEthernet1/1 | include rate
      Queueing strategy: fifo
      5 minute input rate 23357000 bits/sec, 15124 packets/sec
      5 minute output rate 2443000 bits/sec, 306 packets/sec
    
  • host_chost_c Patron Provider, Top Host, Megathread Squad

    Florian,

    Get in touch with RDS NOC, they can apply filters in this case, and can handle DDOS much better than Orange. If you leave the traffic on RDS, at some point their filters will eventually kick in by default.

    Thanked by 1FlorinMarian
  • FlorinMarianFlorinMarian Member, Host Rep

    @host_c said:
    Florian,

    Get in touch with RDS NOC, they can apply filters in this case, and can handle DDOS much better than Orange. If you leave the traffic on RDS, at some point their filters will eventually kick in by default.

    Hey!
    I'll do this but this time Orange has it's own issued.
    I remember that Calin had recently issues with vlan changes and same kind of issue with lot of unexpected traffic.

  • host_chost_c Patron Provider, Top Host, Megathread Squad
    edited January 2024

    @FlorinMarian said: I'll do this but this time Orange has it's own issued.

    Orange has problems at tehnicians, I told you, remember, RDS is better, but not the best.

    @Calin has so much shit on he's network, that even Voxility would tell him to go home :smiley: Sorry bro it is true :D . But yes, he has routing issues at ISP Orange level.

    For the moment, try your luck with RDS NOC, I will PM you their e-mail.

    Thanked by 3FlorinMarian Calin Kris
Sign In or Register to comment.