Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


★ VirMach ★ Fashionably Late Black Friday & Cyber Week 2020 ★ Flash Deals ★ View inside ★ $7/Yr 512M - Page 17
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

★ VirMach ★ Fashionably Late Black Friday & Cyber Week 2020 ★ Flash Deals ★ View inside ★ $7/Yr 512M

1141517192098

Comments

  • @Neoon said: Well, the system needs to verify if a logged in session is still valid, so a query every time does not sound wrong.

    However, the query looks way to complex for just verifying that.

    You can cache the information and invalidate the cache when an action triggers the user status to update.

    Thanked by 1yoursunny
  • @Chronic said: At the scale of WHMCS, one would hope they would be able to afford half-decent programmers.

    Sounds like they imported them from Adobe, given the mess that is Magento and all the Windoze application bloatware, that I've seen over the decades. :|

    Thanked by 1skorous
  • wtongzewtongze Member
    edited December 2020

    @VirMach
    I have purchased one of the flash sales days ago. After I paid the invoice the server status turned from "Pending" to "Cancelled". I haven't receive any information about the server and no one replied to my ticket at all. What should I now? Wait?

    Thanks

  • NeoonNeoon Community Contributor, Veteran

    @Chronic said:

    @Neoon said: Well, the system needs to verify if a logged in session is still valid, so a query every time does not sound wrong.

    However, the query looks way to complex for just verifying that.

    You can cache the information and invalidate the cache when an action triggers the user status to update.

    Make it even more complex, make the query simple, keep that stuff in memory with SQL and you are done.

  • VirMachVirMach Member, Patron Provider

    @Neoon said:
    Well, the system needs to verify if a logged in session is still valid, so a query every time does not sound wrong.
    However, the query looks way to complex for just verifying that.

    Of course if you have a big database, it may suck a lot when you have a slow query getting executed every time someone is logged in.

    It used to be clients, and then sub-accounts. Now it's clients and "users."

    So the fact that it does that is fine and it was when it was just clients, the problem is that it's, now as I understand it, trying to pull from two tables and join them which significantly increases the level of stress on the server. It's no longer going oh cool, this is "Neoon" -- it's doing something like oh this is "Neoon" who is he again? Let me give him a nickname "Neon" and then it does something like this user is "Neon" who was he again? I think we used to call him "Neoon" but let me go through the entire list and compare them to check.

  • @VirMach said:
    I've barely slept

    @VirMach Thank you. There's been a bit of an entitled attitude here lately and I think some people are losing sight of what this is about and the effort that a lot of people are putting into it.

    I look forward to these BF/CM events and your participation is a highlight. Sure there's been some issues and lessons learnt, but it's going to be part of the experience overall.

    Thanks for rolling with it.

    Thanked by 2AlwaysSkint Chronic
  • @VirMach I think you need to place your flash sales and your WHMCS in different corners. Let's just say why not use a system that gives out coupons and let the main traffic flow to it. This way you can cut off the crowds on your main WHMCS server. Potential customers who have received an individual discount code will no longer choke on the WHMCS doors for registration and payment.
    In theory, you can even make it possible to pre-register in this system, and automatically send the data of the registered user from it to the WHMCS common database at the moment when he receives a deal.

    If there is no overload on WHMCS (it looks too gentle) there will be no problem for existing customers.
    I think there are many similar systems out of the box. It will also not be difficult to develop one within your company specifically for your tasks.

    Thanked by 1Chronic
  • Here's a sample, it's not actually the real syntax because Cloudflare won't let me post it.

    select tblusers.*, tblusers_clients.client_id as pivot_client_id, tblusers_clients.auth_user_id as pivot_auth_user_id, tblusers_clients.created_at as pivot_created_at, tblusers_clients.updated_at as pivot_updated_at, tblusers_clients.owner as pivot_owner, tblusers_clients.permissions as pivot_permissions, tblusers_clients.last_login as pivot_last_login from tblusers inner join tblusers_clients on tblusers.id = tblusers_clients.auth_user_id where tblusers_clients.client_id = ? and owner = ? order by tblusers.id asc limit 1

    This happens every time a client is mentioned. So if you're logged in, on every page. Any ticket we open, pretty much anything.

    Is it running MySQL or MariaDB?
    If there is no data that constantly changes (logging last visit time) in the columns then if you have enough free RAM tweaking the query cache may save the week.
    Unfortunately query cache was removed in MySQL 8.0 ... ofc. because it is better to rely on good coding practices than on mysql lotto hits.
    Also it is worth checking if all the rows in join, where, order by, group by are properly indexed. You can't imagine how often it happens that expensive commercial products lack proper indexing.

  • @VirMach said:
    Flash sales are being ended temporarily in the hopes that it calms down. We'll be back with the flash sales on Friday, let's say around 6PM PST.

    We'll update with more information on the flash sale page.

    The client area is still under maintenance. Could you kindly reopen it soon™?

  • yoursunnyyoursunny Member, IPv6 Advocate

    @TomBG said:
    Also it is worth checking if all the rows in join, where, order by, group by are properly indexed. You can't imagine how often it happens that expensive commercial products lack proper indexing.

    When I was an operations intern, we deployed a forum that the company acquired, including a database backup of the existing content and users. The site went live, and it was extremely slow.
    I discovered that the database had no indexes. I had to figure out and add indexes in production when thousands were visiting.

    For MySQL: EXPLAIN command can tell you whether there are missing queries.

  • @VirMach can you help me atleast check my server Invoice 1123695.

    Until now the server not yet delivered.
    Its been 38 Hours since i paid

  • @DianTama said:
    @VirMach can you help me atleast check my server Invoice 1123695.

    Until now the server not yet delivered.
    Its been 38 Hours since i paid

    Mine is about 56 hours, still not heard anything about it. (Invoice #1122705 ) :*

  • @DianTama said:
    @VirMach can you help me atleast check my server Invoice 1123695.

    Until now the server not yet delivered.
    Its been 38 Hours since i paid

    It's fine.
    I'm not Virmach but this is written clearly in the first post, in case you missed:

    Please anticipate your service to potentially not be functional for a few days before you can contact us for a resolution.

    If you were in a hurry, you should not buy a flash sale because a high load is expected.

    Thanked by 2Chronic atindahause
  • tuctuc Member
    edited December 2020

    @VirMach said:
    Here's a sample, it's not actually the real syntax because Cloudflare won't let me post it.

    select tblusers.*, tblusers_clients.client_id as pivot_client_id, tblusers_clients.auth_user_id as pivot_auth_user_id, tblusers_clients.created_at as pivot_created_at, tblusers_clients.updated_at as pivot_updated_at, tblusers_clients.owner as pivot_owner, tblusers_clients.permissions as pivot_permissions, tblusers_clients.last_login as pivot_last_login from tblusers inner join tblusers_clients on tblusers.id = tblusers_clients.auth_user_id where tblusers_clients.client_id = ? and owner = ? order by tblusers.id asc limit 1

    Not sure if such a SQL command, which may be supper slow without using proper indexing when the two tables go large, exists in a system like WHMCS. Most RDBMS usually apply indices for JOIN queries with ORDER, but they are not an AI system to select the optimal indices for each query statement.
    I suggest to modify the SQL command like: …from tblusers USE INDEX(an index on id column) inner join tblusers_clients USE INDEX(some indices on column client_id then column owner) ON…
    In addition, turning one or both of these two tables into memory tables may help.

    Thanked by 1yoursunny
  • Billing system was broken because of Cybermonday and created 6 invoices for a single add funds attempt (never even saw the credit card field).
    I created a ticket to cancel all these invoices, it was replied but I can't read the reply because client area is "under maintenance" for the longest time.
    @VirMach Please cancel these:
    Invoice #1123603
    Invoice #1123604
    Invoice #1123605
    Invoice #1123606
    Invoice #1123607
    Invoice #1123608
    As if losing one hour trying to order a discount vps wasn't bad, I keep losing more time because of it. It's being an expensive non-deal for me, I don't have time for this :(

  • Days ago I asked them to be cancelled, and they were not, today I still get these "This is a notice to remind you that you have an invoice due on 12/01/2020" ... times 6.

  • Panel seem ok now, but please use it only if you need.

  • @VirMach Hi, yesterday's 502 error made me lose the opportunity to pay(i use Alipay, invoice id#1123743). Is possible to reopen the invoice?I already made a ticket (id #608418).

  • Thanks @ben47955 , trying to open ticket but can already see status and says "Awaiting Billing Department" ... :|
    PS. I've seen too many Cloudflare issues past weeks, some of which they never acknowledge but were reported by many people. Not cool, I removed them from some sites.

  • yoursunnyyoursunny Member, IPv6 Advocate

    @webmaster24 said:
    As if losing one hour trying to order a discount vps wasn't bad, I keep losing more time because of it. It's being an expensive non-deal for me, I don't have time for this :(

    Minimal wage is $10.50. Every hour wasted on 524 error means your server becomes $10.50 more expensive than the deal price.

    I spent at least 2 hours grabbing the $8.88 deal, so it now costs $29.88 for me.

    @imok said:

    @DianTama said:
    Until now the server not yet delivered.
    Its been 38 Hours since i paid

    If you were in a hurry, you should not buy a flash sale because a high load is expected.

    Everyone is in a hurry to get a server, run YABS, and idle it for the rest of the year.

    Thanked by 2ariq01 vyas11
  • Great, after logging in is Under Maintenance again.

  • VirMachVirMach Member, Patron Provider

    @Unicom said:
    @VirMach I think you need to place your flash sales and your WHMCS in different corners. Let's just say why not use a system that gives out coupons and let the main traffic flow to it. This way you can cut off the crowds on your main WHMCS server. Potential customers who have received an individual discount code will no longer choke on the WHMCS doors for registration and payment.
    In theory, you can even make it possible to pre-register in this system, and automatically send the data of the registered user from it to the WHMCS common database at the moment when he receives a deal.

    If there is no overload on WHMCS (it looks too gentle) there will be no problem for existing customers.
    I think there are many similar systems out of the box. It will also not be difficult to develop one within your company specifically for your tasks.

    This was planned but we never got to it; we have a lot of ideas that don't make it due to time constraints. Just to be clear though, at this point this appears to be an inherent issue with how WHMCS decided to do things and no matter what, it would not scale well.

    I think this is pretty much proven now that we ended the flash sales but still have to put it in maintenance mode from these queries running from people just performing normal functions. Sure, if we had no sale at all, ever, then the website would just be slightly slow and usable for the existing customers, but that's not realistic either. At some point in time this would have happened, it's just unfortunate that it had to be us updating WHMCS right before the biggest sale of the year.

    @TomBG said: If there is no data that constantly changes (logging last visit time) in the columns then if you have enough free RAM tweaking the query cache may save the week.

    You're right. This was one of the main tweaks we performed, and what happened was that the coding's just so bad that we have to infinitely increase this for it to work with the quantity we're dealing with right now. Query caching works insanely great and it's what we're using to make the admin panel bearable to answer tickets (each ticket counts as one of the queries above.) So we don't have to constantly do it over and over and save about a minute per load of the ticket page, which adds up.

    This is also what made the client area super smooth in testing, but once we reach a critical mass it all goes to hell.

    @TomBG said: Also it is worth checking if all the rows in join, where, order by, group by are properly indexed. You can't imagine how often it happens that expensive commercial products lack proper indexing.

    This is the direction I'm trying to take the conversation with WHMCS right now and I'm pleased to say that we finally have someone competent at WHMCS taking a look at this and taking it seriously rather than only blaming us.

    @dTQzBen8 said: The client area is still under maintenance. Could you kindly reopen it soon™?

    We really can't -- we keep trying but it's best to keep it closed so we can power through all these tickets and resolve issues. If we open it up, then we can't do tickets.

    @DianTama said: @VirMach can you help me atleast check my server Invoice 1123695.

    Until now the server not yet delivered.

    Its been 38 Hours since i paid

    @wtongze said: Mine is about 56 hours, still not heard anything about it. (Invoice #1122705 )

    Given the circumstances, I can pretty much guarantee you that this will take much longer and that we would, unfortunately, have to go through important tickets with existing services and fix those issues first before addressing your specific issues with new purchases.

    It's nowhere near an ideal situation, but the possibility was there and we discussed that as @imok mentioned.

    @tuc said: In addition, turning one or both of these two tables into memory tables may help.

    This is something else we were considering but I honestly am exhausted from all the changes and optimizations made and we're falling behind with everything else so I would rather not venture into that right now.

    Everything that seems so simple turns out to have extreme unintended consequences this week.

  • yoursunnyyoursunny Member, IPv6 Advocate

    It's time to ditch WHMCS.

    • dual 64 core EPYC
    • 1TB RAM
    • 100Gbps network
    • rewrite the billing software in Go <== most important
    • use distributed in-memory database (no disk at all, because you don't ever power off all the replicas at the same time)
  • I can't login Client Area to manage my VPS for days :(

  • @yoursunny said:
    It's time to ditch WHMCS.

    • dual 64 core EPYC
    • 1TB RAM
    • 100Gbps network
    • rewrite the billing software in Go <== most important
    • use distributed in-memory database (no disk at all, because you don't ever power off all the replicas at the same time)
    • more capacity with more push-up <== most important NOW
  • @yoursunny said: dual 64 core EPYC
    1TB RAM
    100Gbps network

    Where can I get this for $7?

    Off-topic: Is it just me or is the Vanilla quoting a bit borked? I swear it used to work better.

  • yoursunnyyoursunny Member, IPv6 Advocate

    @dTQzBen8 said:

    • more capacity with more push-up <== most important NOW

    Wanna contribute your push-ups to https://pushups.ndn.today ?
    I don't have code ready for processing uploads. For now, you can put the video on Dropbox (affiliate link) and send me a share link.

    @Chronic said:

    @yoursunny said: dual 64 core EPYC
    1TB RAM
    100Gbps network

    Where can I get this for $7?

    Limit for dedicated machine is $84.
    I'm not a provider so I can't post the link.

    Off-topic: Is it just me or is the Vanilla quoting a bit borked? I swear it used to work better.

    Quoting works fine for me. Use the preview button.

    Thanked by 1dTQzBen8
  • @yoursunny said: Quoting works fine for me. Use the preview button.

    Only puts a > at the beginning of the first line for me.

  • @Chronic said: Off-topic: Is it just me or is the Vanilla quoting a bit borked? I swear it used to work better.

    One issue I ran into: It seems to put the ">" code for quoting only at the beginning (first line) of the excerpt, so if the post you're quoting has multiple paragraphs/lines, then you have to manually add the "> " to the beginning of lines 2+ for it to work as expected. But yes, I agree with @yoursunny, "Preview" is your friend!...

    Thanked by 1Chronic
  • @VirMach said: Here's a sample, it's not actually the real syntax because Cloudflare won't let me post it.

    select tblusers.*, tblusers_clients.client_id as pivot_client_id, tblusers_clients.auth_user_id as pivot_auth_user_id, tblusers_clients.created_at as pivot_created_at, tblusers_clients.updated_at as pivot_updated_at, tblusers_clients.owner as pivot_owner, tblusers_clients.permissions as pivot_permissions, tblusers_clients.last_login as pivot_last_login from tblusers inner join tblusers_clients on tblusers.id = tblusers_clients.auth_user_id where tblusers_clients.client_id = ? and owner = ? order by tblusers.id asc limit 1

    This happens every time a client is mentioned.

    That query doesn't look too bad, as long as there's indexes on client_id and owner.

    But, it sounds like it's running that query every single time, rather than caching the data? That's crazy. It's not difficult to throw the data into APC or Memcache or something and invalidate the cache when the user's data changes.

    Apparently WHMCS developers do not know what "caching" is, given it's a feature request: https://requests.whmcs.com/topic/whmcs-to-use-caching. Apparently they don't even support opcode caching? Just given that thread, I'm not really confident in their abilities to develop reliable software any more. I didn't realise it was that bad. I rescind my older comment that said "WHMCS is fine for what it was designed for".

    Thanked by 2Chronic yoursunny
Sign In or Register to comment.