Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Namecrane is down ?

2

Comments

  • @Francisco said: we need the room for a new product launch

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    @ypmLA77zcs said:

    ...just the passwords were like re-generated which is fine .

    Fine even though there was no notification this was going to happen, and it required deleting and re-adding all mail accounts into the client? OK...

    Sorry. Blame smartermail. We updated to the newest version over the weekend and reading back on their changelog they did some internal changes that made it regen all app passwords.

    You can check their changelog, this isn’t me speaking outta my butt.

    Francisco

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    @marian said:

    @Francisco said: we need the room for a new product launch

    👀

    Francisco

    Thanked by 2marian COLBYLICIOUS
  • @Francisco what happened to lu-shared02? Is the maintenance over?

    Somehow, yesterday, all the users using the database were not working and needed some modifications in DirectAdmin (no info were sent that some modifications are needed).

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    @Chalipa said:
    @Francisco what happened to lu-shared02? Is the maintenance over?

    Somehow, yesterday, all the users using the database were not working and needed some modifications in DirectAdmin (no info were sent that some modifications are needed).

    Maintenance was done on Friday. It was pretty major requiring a totally new node be built and we sync data over. We tried I want to say ~10 times to elevate over the past year and it just wouldn't' go.

    Its always possible I missed a file on the copy over, but I can check. We didn't wipe the old drive so I can comb through it, just let me know what you're seeing and I'll check it out :)

    Francisco

  • @Francisco said:

    @ypmLA77zcs said:

    ...just the passwords were like re-generated which is fine .

    Fine even though there was no notification this was going to happen, and it required deleting and re-adding all mail accounts into the client? OK...

    Sorry. Blame smartermail. We updated to the newest version over the weekend and reading back on their changelog they did some internal changes that made it regen all app passwords.

    You can check their changelog, this isn’t me speaking outta my butt.

    Francisco

    So you didn't read the changelog ahead of time? Yolo style?

  • jcn50jcn50 Member
    edited November 3

    @ypmLA77zcs said: Fine even though there was no notification this was going to happen, and it required deleting and re-adding all mail accounts into the client? OK...

    Really??........

    @Francisco said: Sorry. Blame smartermail. We updated to the newest version over the weekend and reading back on their changelog they did some internal changes that made it regen all app passwords.

    That is a terrible patch/upgrade.........

  • @TimboJones said:

    @Francisco said:

    @ypmLA77zcs said:

    ...just the passwords were like re-generated which is fine .

    Fine even though there was no notification this was going to happen, and it required deleting and re-adding all mail accounts into the client? OK...

    Sorry. Blame smartermail. We updated to the newest version over the weekend and reading back on their changelog they did some internal changes that made it regen all app passwords.

    You can check their changelog, this isn’t me speaking outta my butt.

    Francisco

    So you didn't read the changelog ahead of time? Yolo style?

    sysadmin is nothing without a little hope and blind clicking

  • @LEBUserJoe said:

    @TimboJones said:

    @Francisco said:

    @ypmLA77zcs said:

    ...just the passwords were like re-generated which is fine .

    Fine even though there was no notification this was going to happen, and it required deleting and re-adding all mail accounts into the client? OK...

    Sorry. Blame smartermail. We updated to the newest version over the weekend and reading back on their changelog they did some internal changes that made it regen all app passwords.

    You can check their changelog, this isn’t me speaking outta my butt.

    Francisco

    So you didn't read the changelog ahead of time? Yolo style?

    sysadmin is nothing without a little hope and blind clicking

    Maybe if you're Mickey Mouse. Anything that is critical like this would have read release notes/changelog and rollback strategy as SOP.

  • @TimboJones said:

    @LEBUserJoe said:

    @TimboJones said:

    @Francisco said:

    @ypmLA77zcs said:

    ...just the passwords were like re-generated which is fine .

    Fine even though there was no notification this was going to happen, and it required deleting and re-adding all mail accounts into the client? OK...

    Sorry. Blame smartermail. We updated to the newest version over the weekend and reading back on their changelog they did some internal changes that made it regen all app passwords.

    You can check their changelog, this isn’t me speaking outta my butt.

    Francisco

    So you didn't read the changelog ahead of time? Yolo style?

    sysadmin is nothing without a little hope and blind clicking

    Maybe if you're Mickey Mouse. Anything that is critical like this would have read release notes/changelog and rollback strategy as SOP.

    imo especially with current state of exploits, should be a lot deeper check than release notes, im just saying reality of 90% of hosts vibeee way before AI

  • ftp service is still not able to use?

  • uhuuhu Member

    There's no mention of the recent mail outage on the NameCrane news / announcements section. There's nothing in the news section on their Discord. There's also nothing about the STARTTLS issue. There was a supplier documented change that regenerated app passwords, and not only didn't they read the changelog before deploying, but there's also no mention of it on their website or Discord news.

    Instead, you have to come here to read about (some of) it. Remind me why this was the recommended alternative to MxRoute?

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    @uhu said:
    There's no mention of the recent mail outage on the NameCrane news / announcements section. There's nothing in the news section on their Discord. There's also nothing about the STARTTLS issue. There was a supplier documented change that regenerated app passwords, and not only didn't they read the changelog before deploying, but there's also no mention of it on their website or Discord news.

    Instead, you have to come here to read about (some of) it. Remind me why this was the recommended alternative to MxRoute?

    You’re making a lot of assumptions, so let me help you a bit.

    We discussed the power outage earlier in this thread and in tickets. We haven’t sent an email since we are still hoping for some sort of RFO. For now we are making our own improvements (splitting rack, etc). Once we have a concrete date we will email customers and do the maintenance during off business hours.

    The starttls issue is a network burp on hurricane electric. We can try favoring the routes to take cogent, but the longer term plan is to to warm up EU relays. We had started that but when I swapped over everyone to mxfilter it undid Mikes work.

    For the app passwords, we had already discussed it in public in September and didn’t rush to update until we had a better gauge of how disruptive it would be. We are talking low single digit total inboxes in the end, the very very very vast majority of users are just using their imap password. We didn’t intend to spur of the moment update, but the update addressed other things we finally needed on our end.

    Francisco

  • uhuuhu Member

    @Francisco said:

    @uhu said:
    There's no mention of the recent mail outage on the NameCrane news / announcements section. There's nothing in the news section on their Discord. There's also nothing about the STARTTLS issue. There was a supplier documented change that regenerated app passwords, and not only didn't they read the changelog before deploying, but there's also no mention of it on their website or Discord news.

    Instead, you have to come here to read about (some of) it. Remind me why this was the recommended alternative to MxRoute?

    You’re making a lot of assumptions, so let me help you a bit.

    We discussed the power outage earlier in this thread and in tickets. We haven’t sent an email since we are still hoping for some sort of RFO. For now we are making our own improvements (splitting rack, etc). Once we have a concrete date we will email customers and do the maintenance during off business hours.

    The starttls issue is a network burp on hurricane electric. We can try favoring the routes to take cogent, but the longer term plan is to to warm up EU relays. We had started that but when I swapped over everyone to mxfilter it undid Mikes work.

    For the app passwords, we had already discussed it in public in September and didn’t rush to update until we had a better gauge of how disruptive it would be. We are talking low single digit total inboxes in the end, the very very very vast majority of users are just using their imap password. We didn’t intend to spur of the moment update, but the update addressed other things we finally needed on our end.

    Francisco

    You’re making a lot of assumptions, so let me help you a bit.

    Customers want to know what is going on. Even if you don't have the full information, communication is key. The first place people look is on your website. Not Discord. Not here.

    Thanked by 1analog
  • @uhu said:

    @Francisco said:

    @uhu said:
    There's no mention of the recent mail outage on the NameCrane news / announcements section. There's nothing in the news section on their Discord. There's also nothing about the STARTTLS issue. There was a supplier documented change that regenerated app passwords, and not only didn't they read the changelog before deploying, but there's also no mention of it on their website or Discord news.

    Instead, you have to come here to read about (some of) it. Remind me why this was the recommended alternative to MxRoute?

    You’re making a lot of assumptions, so let me help you a bit.

    We discussed the power outage earlier in this thread and in tickets. We haven’t sent an email since we are still hoping for some sort of RFO. For now we are making our own improvements (splitting rack, etc). Once we have a concrete date we will email customers and do the maintenance during off business hours.

    The starttls issue is a network burp on hurricane electric. We can try favoring the routes to take cogent, but the longer term plan is to to warm up EU relays. We had started that but when I swapped over everyone to mxfilter it undid Mikes work.

    For the app passwords, we had already discussed it in public in September and didn’t rush to update until we had a better gauge of how disruptive it would be. We are talking low single digit total inboxes in the end, the very very very vast majority of users are just using their imap password. We didn’t intend to spur of the moment update, but the update addressed other things we finally needed on our end.

    Francisco

    You’re making a lot of assumptions, so let me help you a bit.

    Customers want to know what is going on. Even if you don't have the full information, communication is key. The first place people look is on your website. Not Discord. Not here.

    I migrated from Mxroute to Namecrane mail services and having this outgage which i am sure its not of @Francisco hands . he is trying his best and to acheive good customer service.

    Pretty sure things will get better and he will give us new offers pretty soon .

  • @William said:
    pure luck, i dont know more either, but once transatlantic links started to go haywire yesterday (traffic on some went up 1000%, see the OVH weathermap) i centralised all my shit to Norway finally

    My NL services (worldstream) are still fine.

    Who did you migrate to (Norway)?

  • @amsaal said:

    @uhu said:

    @Francisco said:

    @uhu said:
    There's no mention of the recent mail outage on the NameCrane news / announcements section. There's nothing in the news section on their Discord. There's also nothing about the STARTTLS issue. There was a supplier documented change that regenerated app passwords, and not only didn't they read the changelog before deploying, but there's also no mention of it on their website or Discord news.

    Instead, you have to come here to read about (some of) it. Remind me why this was the recommended alternative to MxRoute?

    You’re making a lot of assumptions, so let me help you a bit.

    We discussed the power outage earlier in this thread and in tickets. We haven’t sent an email since we are still hoping for some sort of RFO. For now we are making our own improvements (splitting rack, etc). Once we have a concrete date we will email customers and do the maintenance during off business hours.

    The starttls issue is a network burp on hurricane electric. We can try favoring the routes to take cogent, but the longer term plan is to to warm up EU relays. We had started that but when I swapped over everyone to mxfilter it undid Mikes work.

    For the app passwords, we had already discussed it in public in September and didn’t rush to update until we had a better gauge of how disruptive it would be. We are talking low single digit total inboxes in the end, the very very very vast majority of users are just using their imap password. We didn’t intend to spur of the moment update, but the update addressed other things we finally needed on our end.

    Francisco

    You’re making a lot of assumptions, so let me help you a bit.

    Customers want to know what is going on. Even if you don't have the full information, communication is key. The first place people look is on your website. Not Discord. Not here.

    I migrated from Mxroute to Namecrane mail services and having this outgage which i am sure its not of @Francisco hands . he is trying his best and to acheive good customer service.

    Pretty sure things will get better and he will give us new offers pretty soon .

    "I migrated from trash, to a service that was offline immediately, instead of judging my new provider I gargled on his big penis and told everyone to relax! i'm new here I know more than you 3+ year customer's he is doing his best! it's totally not a for profit, that targets business + clientel and should have much better uptime, communication, support, and development NAHHHH! but i'll hold MXROUTE TO IT!

    u liberals

    Thanked by 3zed uhu analog
  • FranciscoFrancisco Top Host, Host Rep, Veteran

    @uhu said: Customers want to know what is going on. Even if you don't have the full information, communication is key. The first place people look is on your website. Not Discord. Not here.

    When we did restoration we handled customer facing gear first. Once all the shared/mail servers were back online we then worked to get the corporate side back up. Corporate side is a bit more involved due to ceph and such.

    Customers that ticketed got full info. An external 'Status' page would be useful and i'll work on implementing something like that. Again, during this whole thing I was the only one on shift/awake,
    so my priority was to get things spun back up with me giving quick notes to discord as I moved around.

    Thanks for the feedback, i'll take it to heart and see where we can improve on it :)

    Francisco

    Thanked by 2uhu BasToTheMax
  • uhuuhu Member

    @Francisco said:

    @uhu said: Customers want to know what is going on. Even if you don't have the full information, communication is key. The first place people look is on your website. Not Discord. Not here.

    An external 'Status' page would be useful and i'll work on implementing something like that.

    Yeah, top tip for all providers, don't host your own status page in the same DC as your services.

    Again, during this whole thing I was the only one on shift/awake,

    Understandable. The best support status page I've seen is a very simple "We are aware of the following issues: ... " with timestamps. Otherwise, you get what we had here, which pretty much boiled down to "Are they even aware of the problem" and "Do they care?"

    The answer to both can be "Yes", but if the customer can't see that, they assume "No".

  • @longtable2650 said:

    @William said:
    pure luck, i dont know more either, but once transatlantic links started to go haywire yesterday (traffic on some went up 1000%, see the OVH weathermap) i centralised all my shit to Norway finally

    My NL services (worldstream) are still fine.

    Who did you migrate to (Norway)?

    Gigahost, but i had AI set up a locked down vhost on my VPS for wordpress only as its my only non-python app.

  • ypmLA77zcsypmLA77zcs Member
    edited November 4

    @Francisco said:
    ...
    Customers that ticketed got full info
    ...

    Thanks for the feedback, i'll take it to heart and see where we can improve on it :)

    Francisco

    My ticket # 29677 opened Nov. 1st has not been answered still

  • ypmLA77zcsypmLA77zcs Member
    edited November 4

    @Francisco said:
    ...
    For the app passwords, we had already discussed it in public in September and didn’t rush to >update until we had a better gauge of how disruptive it would be. We are talking low single >digit total inboxes in the end, the very very very vast majority of users are just using their >imap password. We didn’t intend to spur of the moment update, but the update addressed >other things we finally needed on our end.

    Francisco

    IMAP password is an app password (thus re-generated), why would you say it wasn't affected?

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    "It depends".

    If a user has 2FA enabled then yes, IMAP has an 'app password'. If they don't then their IMAP password == their webmail and everything else. The amount of people with 2FA, and thus "app passwords", is in the low single digit percents.

    @ypmLA77zcs said: My ticket # 29677 opened Nov. 1st has not been answered still

    I can take a look, that might be in my ticket queue. The 1st would've been Sunday when we were sorting things.

    Francisco

  • down again it seems...says i have to verify email address when signing in..and all pages return a 404..
    i also hope that the anti spam for emails is not deleting them but putting them in junk mail. i also had to buy the package like 5 times because i was flagged for fraud. tried several ips even my own isp one and 2 different sim cards on cellular data. my secondary sim got through. even though i'm member on the previous site for a long time.
    i hope these problems stop soon and that i didnt buy the lifetime package for a service that goes down often....

  • looks like its back just now. maybe some moving from old to new design. new design looking good.

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    @borgqueenx said:
    down again it seems...says i have to verify email address when signing in..and all pages return a 404..
    i also hope that the anti spam for emails is not deleting them but putting them in junk mail. i also had to buy the package like 5 times because i was flagged for fraud. tried several ips even my own isp one and 2 different sim cards on cellular data. my secondary sim got through. even though i'm member on the previous site for a long time.
    i hope these problems stop soon and that i didnt buy the lifetime package for a service that goes down often....

    We sent an email about us doing the new site and billing portal :) It took a few hours longer, but we decided to do some extra work along the way.

    I'll check the verification emails, some others said they weren't getting them so i'm wondering if its hitting a rate limit by accident.

    Francisco

  • suutsuut Member
    edited November 7

    The website has undergone a major overhaul and feels completely next-level.

    Thanked by 1averagedatahoarder
  • no more DA shared hosting?

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    @cloromorpho said:
    no more DA shared hosting?

    Nope, no real reason to offer both portals. All it does is cause confusion and more tickets. The new plans are quite meaty.

    Francisco

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    @suut said:
    The website has undergone a major overhaul and feels completely next-level.

    Thanks :) Have you found the crane game yet? Or got snake eyes?

    Francisco

    Thanked by 2suut bikegremlin
Sign In or Register to comment.