Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


HAZI.ro | Performance drops expected tomorrow for VPSs in Romania - Page 16
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

HAZI.ro | Performance drops expected tomorrow for VPSs in Romania

1131416181932

Comments

  • CalinCalin Member, Patron Provider
    edited January 9

    @FatStacks said: After 15 pages he still doesn't understand the purpose of a DDoS protection that filters the traffic BEFORE entering your network.

    >

    Before entering your own network it costs a lot, and we have encountered a similar problem in the past

    @FlorinMarian I recomanded you to no take the Fortigate solution , I'm talk with other friends , I offered them what specifications give orange for the fortigate,and it's just a bullshit

    Custom solutions can be made, but for that you need physical access to the backbone and the orange network, orange accepts this but there is a long list of conditions, first of all the person must be authorized with several certificates, equipment cost (because it is custom ) and many others, prices start from 20k+++ euros(just configuration),+ month payment for custom solution(i m speak here about corporate prices) something we didn't know would work, so we canceled this project, anti-ddos protection is a very risky game

    besides the fact that I didn't have 20k euros to invest, I didn't even know if it would work or not

    Regards

  • tentortentor Member, Host Rep

    @FlorinMarian said: If I use GRE tunnel, when the attacker targets ISP's IP address, GRE is 0.

    ISP can establish the BGP session over RFC1918 IP addresses or give you an IP address from a prefix which is useful only within AS ("no-export" community)

    Thanked by 1RapToN
  • FlorinMarianFlorinMarian Member, Host Rep

    @tentor said:

    @FlorinMarian said: If I use GRE tunnel, when the attacker targets ISP's IP address, GRE is 0.

    ISP can establish the BGP session over RFC1918 IP addresses or give you an IP address from a prefix which is useful only within AS ("no-export" community)

    Any such modification takes up to 45 days.
    I've asked them to change public IPs to private ones and even for this they require a discussion with commercial consultant, extra payment and up to 30 days deployment time. (about 15 days takes until you get the documents to be signed).

  • tentortentor Member, Host Rep

    @FlorinMarian said: Any such modification takes up to 45 days.

    Skill ISP issue. Don't you have any better alternatives in Romania?

    Thanked by 2yoursunny doghouch
  • CalinCalin Member, Patron Provider
    edited January 9

    @FlorinMarian said: I've asked them to change public IPs to private ones and even for this they require a discussion with commercial consultant, extra payment and up to 30 days deployment time. (about 15 days takes until you get the documents to be signed).

    >

    Welcome to the hosting industry :) again , this it's just benginer , I'm very happy when you start own more servers , 50-60+

    Meet problems with

    Electricity (the most big for us , and what meet lot of problems ,because, when you use approximately 18 KWh/hour 24/7, the cables do not have time to cool down,
    ) and this need lot of solutions , both externally and internally,and moooore lot of problems
    Cooling
    Noise
    UPS Overloaded (Need rooms for fix this problems and other other other things)
    ISU Autorisation (yes yes , after make 9-10 MWs of month electricity) you electric supply start ask what the f**k you make on you house , what usage 10+ MWs/month electricity

    Etc..etc...etc...

    Regards

  • AndreixAndreix Member, Host Rep
    edited January 9

    @Calin said:
    Welcome to the hosting industry :)

    That's not the "hosting industry", we have a more appropriate word for what you two are doing: bișniță [1].
    If you live under the impression that you are a part of the Romanian (or even worse, world) hosting industry, I'm sorry to wake you up, but you're not.

    [1] https://dexonline.ro/definitie/bișnițar

  • CalinCalin Member, Patron Provider

    I m start laught @Andreix :)))

  • CalinCalin Member, Patron Provider

    @tentor said: Skill ISP issue. Don't you have any better alternatives in Romania?

    >

    Nop , in general any things what need lot of time and birocracy , and here I'm not speak just for ISP , in general , for all things

    Regards

  • AndreixAndreix Member, Host Rep
    edited January 9

    @Calin said:
    I m start laught @Andreix :)))

    You should.
    Romanian hosting industry is composed by providers, not script kiddies that have to google how to install apache with yum.
    Providers like: Romarg, MXHost, EasyHost (ex. Hostway Romania), MXHost, SimpliQ + Webfactor, Hostico, ClausWeb, Clues (now M247), Hosterion... and a handful of others.

    Thanked by 1cioan
  • CalinCalin Member, Patron Provider
    edited January 9

    @Andreix said: Webfactor

    Webfactor? Really? You joke or? You don't know the story of the background of webfactor :)) webfactor it's very similar at what we have my and florinmarian

    that's why he went bankrupt and after this other 2x companys purchase company in just 4 years , first simpliq and after this tennet

  • AndreixAndreix Member, Host Rep
    edited January 9

    I know the story of Webfactor and I still put it in the provider category and 100% was an important factor in the hosting industry evolution in Romania.
    Of course, they had their downfall, but the approach was totally different by negotiating with Simpliq DC to take over all affected customers (and the whole business in the end), not by leaving all infra down thinking "they can start over any time".

    When you'll have the knowledge and infra to host a bank's infra, for example, you could be a called a provider. However, honestly, I doubt it will happen in the next 25 years.

  • CalinCalin Member, Patron Provider

    @Andreix said: they can start over any time

    First , my never i'm saying this , I'm definitely not that kind of person, I generally put a lot of soul into a project

    @Andreix said: When you'll have the knowledge and infra to host a bank's infra, for example, you could be a called a provider. However, honestly, I doubt it will happen in the next 25 years.

    >

    Yeah that it's true but heyy , there are several types of hosting in this industry, for example I am on the backup side, those who host banks are on the corporate side etc... etc...

    Regards

  • AndreixAndreix Member, Host Rep

    @Calin said:

    @Andreix said: they can start over any time

    First , my never i'm saying this , I'm definitely not that kind of person, I generally put a lot of soul into a project

    @Andreix said: When you'll have the knowledge and infra to host a bank's infra, for example, you could be a called a provider. However, honestly, I doubt it will happen in the next 25 years.

    >

    Yeah that it's true but heyy , there are several types of hosting in this industry, for example I am on the backup side, those who host banks are on the corporate side etc... etc...

    Regards

    Yeah, we're usually calling those projects: hobbies. The kind of sellers (not providers) that you expect to have a lifespan of open invocie, complete the payment then ERR_SERVER_NOT_FOUND.

  • CalinCalin Member, Patron Provider

    @Andreix said: I know the story of Webfactor

    I don't know you meet , and I know the problems behind the web industry in Romania
    I spoke with several people who worked at webfactor in the past, and they are definitely not a standard hosting company, nor is simpliq

    I'm meet with cristian haja , pambuccian and others....

  • AndreixAndreix Member, Host Rep

    @Calin said:

    I'm meet with cristian haja , pambuccian and others....

    I've worked with few of them, however, I will not list their names here for privacy reasons.
    I am actively implied in the DC/Hosting area since 2009 if I'm not mistaken.

    Thanked by 1host_c
  • @Andreix said:

    @Calin said:
    I m start laught @Andreix :)))

    You should.
    Romanian hosting industry is composed by providers, not script kiddies that have to google how to install apache with yum.
    Providers like: Romarg, MXHost, EasyHost (ex. Hostway Romania), MXHost, SimpliQ + Webfactor, Hostico, ClausWeb, Clues (now M247), Hosterion... and a handful of others.

    If this was a Romanian forum hazi will be destroyed in 1 thread by WBF, rip.

  • AndreixAndreix Member, Host Rep

    @AndreiGhesi said:

    @Andreix said:

    @Calin said:
    I m start laught @Andreix :)))

    You should.
    Romanian hosting industry is composed by providers, not script kiddies that have to google how to install apache with yum.
    Providers like: Romarg, MXHost, EasyHost (ex. Hostway Romania), MXHost, SimpliQ + Webfactor, Hostico, ClausWeb, Clues (now M247), Hosterion... and a handful of others.

    If this was a Romanian forum hazi will be destroyed in 1 thread by WBF, rip.

    Oh, yeah, gazduire.info old times... damn you brought up that nostalgia.
    I'm gonna need a glass of Bumbu.

    Thanked by 1host_c
  • coldcold Member

    @Levi said:

    @FlorinMarian said:
    Small update here

    Good for you! 3 days to suffer and you are good to go. How much extra they asked?

    only until they are done with the work, could be BTW now and Christmas..

  • xrzxrz Member

    @FlorinMarian said: Even a 40Gbps link gets saturated easily.

    get 2 x 1 Tbps, should handle that :D :D

  • KrisKris Member

    @FlorinMarian said: Even a 40Gbps link gets saturated easily.

    If you're really getting attacked, This is why you use Anycast protection + BGP.

    Instead of one location, say Orange Arbor filtering center, your IP is announced to 20 places around the world.

    Each of these 20 places has 40+ Gbps connectivity at least, often more over peering IX's, and acts like a sponge to absorb the DDoS traffic, cleaned in real time, and returned over GRE tunnel to you with clean traffic.

    IT CAN BE DONE. Don't throw them 700 euros per month for one location with an Arbor device when XDP + eBPF can be leveraged at 20, and return 1Gbps clean traffic to you.

    @FlorinMarian said: If I use GRE tunnel, when the attacker targets ISP's IP address, GRE is 0.

    Look into MPLS and TTL manipulation.

    Ever MTR or try to traceroute a DDoS protected provider? You'll often see three sets of non-responding hops towards the end. This is standard practice and achievable pretty easy in terms of hiding ISP hops.

    A VPS with a provider I have here has MPLS outbound too, hiding the first few hops outbound, so when I MTR a DDoS protected provider, you only see your gateway, 6 non-answering hops, and the destination.

    Still not convinced this is a DDoS rather what @host_c and @Calin mentioned. Something happened during maintenance that screwed your Orange connection. Until I see netflow or fastnetmon otherwise, raw packet numbers aren't doing it. That's likely the misconfiguration flailing around.

    Another reason fastnetmon should've been setup days ago with your router pointing to it to collect real stats, but I'm gonna see myself out at this point, you seem to be realizing something's wrong with the connection, but haven't tried riding it out with RCS / RDS for more than a few mins to see if it's truly a DDoS, or if you'd be better lived off a single homed connection until Orange fixes the line.

  • AlexBarakovAlexBarakov Patron Provider, Veteran

    @Kris said: Another reason fastnetmon should've been setup days ago with your router pointing to it to collect real stats, but I'm gonna see myself out at this point, you seem to be realizing something's wrong with the connection, but haven't tried riding it out with RCS / RDS for more than a few mins to see if it's truly a DDoS, or if you'd be better lived off a single homed connection until Orange fixes the line.

    Absolutely second this. FastNetMon should have been set up days ago, when it all started, if not even before that. It would have given a lot more information about the actual attack, the protocols, the size, the targets and everything that's reaching the equipment. Even if used only for traffic telemetry, it's an amazing tool and Pavel (the lead Dev) is also great and helpful. For so little traffic, I'd personally set it in a port mirror mode to get the most accurate and fastest information.

  • FlorinMarianFlorinMarian Member, Host Rep

    @Kris said:

    @FlorinMarian said: Even a 40Gbps link gets saturated easily.

    If you're really getting attacked, This is why you use Anycast protection + BGP.

    Instead of one location, say Orange Arbor filtering center, your IP is announced to 20 places around the world.

    Each of these 20 places has 40+ Gbps connectivity at least, often more over peering IX's, and acts like a sponge to absorb the DDoS traffic, cleaned in real time, and returned over GRE tunnel to you with clean traffic.

    IT CAN BE DONE. Don't throw them 700 euros per month for one location with an Arbor device when XDP + eBPF can be leveraged at 20, and return 1Gbps clean traffic to you.

    @FlorinMarian said: If I use GRE tunnel, when the attacker targets ISP's IP address, GRE is 0.

    Look into MPLS and TTL manipulation.

    Ever MTR or try to traceroute a DDoS protected provider? You'll often see three sets of non-responding hops towards the end. This is standard practice and achievable pretty easy in terms of hiding ISP hops.

    A VPS with a provider I have here has MPLS outbound too, hiding the first few hops outbound, so when I MTR a DDoS protected provider, you only see your gateway, 6 non-answering hops, and the destination.

    Still not convinced this is a DDoS rather what @host_c and @Calin mentioned. Something happened during maintenance that screwed your Orange connection. Until I see netflow or fastnetmon otherwise, raw packet numbers aren't doing it. That's likely the misconfiguration flailing around.

    Another reason fastnetmon should've been setup days ago with your router pointing to it to collect real stats, but I'm gonna see myself out at this point, you seem to be realizing something's wrong with the connection, but haven't tried riding it out with RCS / RDS for more than a few mins to see if it's truly a DDoS, or if you'd be better lived off a single homed connection until Orange fixes the line.

    I did mirroring and there are indeed DDoS attacks.
    The target is the entire Romanian subnet through two methods:

    • IP spoofing (I also found packages from the UK government)
    • DNS Amplification (tens of thousands of parallel servers connected to my IPs)
      At this moment I have to take some very difficult decisions because:

    • if I do nothing, I will be without clients and I will pay for two internet lines in Romania unnecessarily for 2 years.

    • if I buy Arbor, I assume some very high monthly costs as an obligation for 2 years, and the question arises: if it doesn't last?
    • the protection through Anycast costs a lot considering the amount of monthly traffic that I expect
    • to move everything to OVH, I will pay for two internet lines in Romania unnecessarily for 2 years.
  • LeviLevi Member

    @FlorinMarian said:

    @Kris said:

    @FlorinMarian said: Even a 40Gbps link gets saturated easily.

    If you're really getting attacked, This is why you use Anycast protection + BGP.

    Instead of one location, say Orange Arbor filtering center, your IP is announced to 20 places around the world.

    Each of these 20 places has 40+ Gbps connectivity at least, often more over peering IX's, and acts like a sponge to absorb the DDoS traffic, cleaned in real time, and returned over GRE tunnel to you with clean traffic.

    IT CAN BE DONE. Don't throw them 700 euros per month for one location with an Arbor device when XDP + eBPF can be leveraged at 20, and return 1Gbps clean traffic to you.

    @FlorinMarian said: If I use GRE tunnel, when the attacker targets ISP's IP address, GRE is 0.

    Look into MPLS and TTL manipulation.

    Ever MTR or try to traceroute a DDoS protected provider? You'll often see three sets of non-responding hops towards the end. This is standard practice and achievable pretty easy in terms of hiding ISP hops.

    A VPS with a provider I have here has MPLS outbound too, hiding the first few hops outbound, so when I MTR a DDoS protected provider, you only see your gateway, 6 non-answering hops, and the destination.

    Still not convinced this is a DDoS rather what @host_c and @Calin mentioned. Something happened during maintenance that screwed your Orange connection. Until I see netflow or fastnetmon otherwise, raw packet numbers aren't doing it. That's likely the misconfiguration flailing around.

    Another reason fastnetmon should've been setup days ago with your router pointing to it to collect real stats, but I'm gonna see myself out at this point, you seem to be realizing something's wrong with the connection, but haven't tried riding it out with RCS / RDS for more than a few mins to see if it's truly a DDoS, or if you'd be better lived off a single homed connection until Orange fixes the line.

    I did mirroring and there are indeed DDoS attacks.
    The target is the entire Romanian subnet through two methods:

    • IP spoofing (I also found packages from the UK government)
    • DNS Amplification (tens of thousands of parallel servers connected to my IPs)
      At this moment I have to take some very difficult decisions because:

    • if I do nothing, I will be without clients and I will pay for two internet lines in Romania unnecessarily for 2 years.

    • if I buy Arbor, I assume some very high monthly costs as an obligation for 2 years, and the question arises: if it doesn't last?
    • the protection through Anycast costs a lot considering the amount of monthly traffic that I expect
    • to move everything to OVH, I will pay for two internet lines in Romania unnecessarily for 2 years.

    Go cociu after dumping all assets on olx.ro. Orange will chase wind in the fields.

  • xrzxrz Member
    edited January 9

    @Kris said: Still not convinced this is a DDoS

    https://lowendtalk.com/discussion/comment/3872282/#Comment_3872282

    This is just 1 second (7 tcpdump files? idk what that means anyway, 7 various monitored ports ?):
    2400 various IP addr., so that is normal for that small hosting right? i know some bigger sites with visits is normal, but this is not okay...

  • KrisKris Member
    1. Currently what you are doing, it will result in zero clients, and you have two-unnecessary lines for two years.

    2. If you buy Arbor filtering, you're going to find out in a real attack Unicast protection isn't enough when you're knocked offline next time.

    3. Ride it out with a provider like Path, GSL, whoever @dbContext mentioned.

    4. If you move everything to OVH you'd be protected.

    5. You put in announcements on your site after you were going to BYOIP all to OVH, you decided not to, for a technical reason, without stating more, and saying you did not want to - Can the catalyst not handle the session?

    You can setup fastnetmon and if you see specific targets ask your DC to blackhole them for you, or ask for BGP communities they claim to support, without cost.

    @FlorinMarian said: the protection through Anycast costs a lot considering the amount of monthly traffic that I expect

    Yes, it's like saying an Umbrella costs a lot of money, until you're standing a week in a hurricane without protection. Then suddenly it makes sense.

    I'd personally get a DDoS protected Romanian VM. If you expect so little traffic, make a DDoS mitigation service fully symmetrical (outbound too) so you can have complete control, spoofing and Layer 7 protection. The money you will make selling DDoS protected VMs should make up the 500 euros per month easy.

    You still should have gotten a Voxility dedicated a few days ago and offered them some cash to expedite the LOA, then GRE tunnel that back to your place. It would have been slap-dash, but more than nothing. Really is true of any provider that offers DDoS protection within 20 (or 50 ms) of you. Offer OVH say 250 eur to expedite IP announcements, and then try 500 eur if that doesn't work. Final thoughts, but the worst thing you could be doing right now is nothing. It's clear the Orange line is the issue.

    Get a BGP Anycast protection provider and ride it out via RCS/RDS. Also @host_c gave you great advice, sick a lawyer on them to revert your old setup as clearly (from your own troubleshooting with Mom at NOC) there's a lingering network issue, meaning 40% packet loss even without an attack...

    All the best, and if you get DDoS protection through a provider, I'll be one of the first to get a VM to offset it, but not until you change your ways, and start to care about the customer.

    No honor in throwing in the towel and blaming lead times or providers when you have so many other options, many of them free like Fastnetmon that could give you a better look beyond raw packets into this.

    Thanked by 2FlorinMarian adly
  • @FlorinMarian said:

    • if I do nothing, I will be without clients and I will pay for two internet lines in Romania unnecessarily for 2 years.

    Lol. If you will be in position to get bankrupt because of this flood/problem you close the company and the contract is null.

    I hope that you did not made the internet contract on your personal name.

    Thanked by 2yoursunny apollo15
  • host_chost_c Member, Patron Provider

    @FlorinMarian

    I say Orange Fucked up, @Calin say the same thing an he has multiple links from them and multiple problems and he's network is configured by orange all in.

    @Kris tends to agree with me, some think the same.

    After all this is done, probably by page 25-27 and 15K views, if Orange fuked up can we agree that they are the worst in RO. I mean, not even Vodafone managed to do shit like this, nor RDS-RCS.

    Again, Almost 1 week and you are still out-of-band.

    Even the solution with a damn pFsense would have saved you some debugging by now, and you have the hardware for that, I told you, I can send even a Chelsio T520CR card.

    Thanked by 1FlorinMarian
  • CalinCalin Member, Patron Provider

    @AndreiGhesi said: close the company and the contract is null

    Tell this thing at @cociu :)) not that work... it's not a normal contract it's a business contract

    Regards

  • FlorinMarianFlorinMarian Member, Host Rep

    @AndreiGhesi said:

    @FlorinMarian said:

    • if I do nothing, I will be without clients and I will pay for two internet lines in Romania unnecessarily for 2 years.

    Lol. If you will be in position to get bankrupt because of this flood/problem you close the company and the contract is null.

    I hope that you did not made the internet contract on your personal name.

    I talked to the accountant and it's not like that at all.
    To enter insolvency, you must not pay your suppliers for 3-6 months, find a good debtor (who will sell the company's assets and pay as much as possible of the debts), things far too complicated to consider a new SRL to take over the activity of the current one.
    Having someone sell my servers and the investments made for the benefit of my parents is the last thing I take into account.

  • xrzxrz Member

    @host_c said: I say Orange Fucked up, @Calin say the same thing an he has multiple links from them and multiple problems and he's network is configured by orange all in.

    the truth maybe lies somewhere else probably (pissed someone? coincidence?):
    https://lowendtalk.com/discussion/comment/3858891/#Comment_3858891

Sign In or Register to comment.