New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
Oops yes you're right it was Invision rather than Xenforo. But iirc it wasn't a switch to Invision that made people leave. It was an upgrade from an older version to a newer version of the same Invision product, that was required in order to keep getting vendor support. But the new version was much less usable than the old version.
Yeah the newer Invision Board upgrade killed it - search/new topics was bad. My frequented forums I usually bookmark the new thread/topics link and come into forum visits via them where possible
You could probably make it work, but you'll put yourself in a world of pain. Ultimately, it somewhat depends on how you do it, but you'll need to consider cache invalidation whenever a user changes their avatar/signature, admin edits templates/theme (which can also be problematic if you allow multiple user-selectable themes) and any dynamic content will be problematic (e.g. a download counter for attachments). If you display user post counts, for example, that'll also be painful, and if users can select their own timezone, showing the correct post date/time without Javascript can be a problem.
You can probably work around these, but be aware that you'll seriously limit what you can show alongside posts, and it'll make customizations/modifications much more difficult to get right.
If you decide to take the 'many cache invalidations' route, that has its own set of problems. Either you're very fine grained with invalidations (e.g. user changes avatar -> invalidate all pre-rendered posts for that user), which makes invalidation itself somewhat of an exercise, or you do splats (e.g. change avatar -> invalidate all posts on forum) which lessens the effectiveness of the cache and can be problematic on high traffic boards due to contention.
Optimizations are cool, but don't go crazy on them. In this case, it probably isn't worth the hassle.
The
gziputility does this, but I don't think zlib does, so it likely won't work. If it doesn't, you could instead concatenate deflate blocks to achieve more or less the same result though.But clearing dictionary/entropy contexts every ~1.5KB is likely going to seriously impact the compression efficiency you'll get, particularly if you intend to serve hundreds of posts per page. To be honest, it would make more sense to compress the entire page/thread and cache that.
99% of these cases are probably the admins not knowing how to configure servers properly, using horribly inefficient plugins/modifications and/or just simply wanting way more than they need ("for if our forum explodes in popularity one day") or not understanding their requirements properly ("my site is 'high traffic' so I need the best configuration available").
I'm also always suspicious of sites which ask for donations but don't show numbers. "Server costs" are a convenient justification for all sorts of things.
This mostly comes from my own experience with running forums years ago - I can't remember all the figures correctly, but it was something like this. For a single banner ad, I got a CPM rate of US$2 and with only a 10% fill rate, 40k pageviews/day, around $5/day or ~$150/month (after ad blockers). 40k pageviews/day on a not really optimized PHP/MySQL forum easily ran on a $15/year LE VPS at the time, assuming the server is somewhat tuned. Doing the math, you can see that server costs barely enter the equation, and from an earnings perspective, you're much better off trying to increase traffic than you are trying to optimise costs. Though looking at forums with similar traffic profiles to mine, they'd often be running on significantly more expensive servers (usually 10x the cost), but even then, costs aren't really much of an issue.
Now of course, that's my case - perhaps people want better reliability than a cheap LEB, maybe they use AWS because of prestige, so higher costs. But I suspect that a 10% fill rate is quite low (have no idea what the standard is, but I didn't really try to optimize earnings) so there's potential there. On the other hand, maybe CPM rates are lower these days due to more competition etc. But I find it hard to believe that server costs to be that much of a problem for anyone who can run a forum somewhat well, and willing to display ads*.
* of course, there are admins who don't want to display ads, and may even run on zero revenue, so costs are the only thing that can be optimized. Still, with today's servers, it shouldn't be an expensive endeavour as long as the software isn't too horrible.
reminds me of another feature for forums supporting edge side includes or fragmented caching of some form - yeah hard to do but would be nice to have
As to server costs, really depends on size of forums too. I know some forums with up to 450GB attachments and one even had 4.7TB of attachments! Just spec'ing a server to host such a forum + backup costs isn't cheap even if you do/don't have the traffic numbers to pull ad revenues to cover such costs.
Those "forums" probably had big legal bills too.
I dunno... attachments typically are just blobs of data that people download, and don't need fast random disk access (unlike a DB), so a bunch of cheap spinning disks often does the trick. Backups in particular don't need to be on fast storage either.
To me, I'd think that bandwidth would be more of the issue with forums which host large amounts of attachments, not disk space. If this "forum" is really more of a download/file sharing site, just in a forum format (or using a forum script as a base), then the cost equation is a little different to typical forums, and more on that of a file sharing site.
Attachments need to be deleted regularly in my experience. Unless you are a big op, it doesn't make sense to keep them alive all the time.
You guys wrote a lot of good stuff about the technical aspect of things, however, I think that the answer should also include the, unexpectedly, not so much talked about side of product development - the actual work. So with every project, I try to give myself an honest answer to 3 starting questions:
To make a good software product you have to work, work, and work. From the most basic features that the end users won't even ever see, to the advanced features that require weeks in planning and hundreds of man-hours to actually write the code. And unfortunately, not a single line of code will write itself. It requires a lot of dedication, a lot of nerves of steel and a lot of hard persistence. In this thread alone there are already a couple of dozens of features that you would have to develop. Nowadays, users are accustomed to having "all the features", which means that you first have to develop every feature that you competitor software already has and then add even more features on top of that, to lure in the users. No one will migrate to your software if they are missing something their current software has, no matter how stupid that feature may seem to you personally. So yeah, if you are not ready to push 70 work hours per week for the first 2 years and then 40h/week for the next 3-5 years, you can bet your competitors for sure are ready, and they already have the upper hand by having the existing user/customer base, most of the features developed and some kind of funding coming in.
You are planning to develop a forum software. It does not belong under the small in size type of software, so the "code quick" method will not help here. You may at first think to yourself "I'll cut everything down to a bare minimum and just make it simple", but this is not how it works in reality because the deeper you get into the development the more "can't do without" features emerge. All of this takes a lot of your time and sometimes you'll be working 12-16h per day. That leaves you with little to no time to do anything else in your life. You really must ask yourself if the end goal is really worth 3-5 years of your life? Sure, it's not about the destination, it's about the journey, but if you're willing to sacrifice so much time you might as well be sure to sacrifice it for the right thing because rarely you'll get the second chance (life is too short).
Users/customers don't actually care about your hard work and your software. All they care about is your software satisfying their needs. Need is the number one driver for them to use your software and if there is no need or the need is already filled, you'll have to come up with something really really good and unique, or the users will never make the necessary switch. You can succeed in building a software that fills needs by applying the "scratch your own itch" method. In your case, this means that you would need to identify what "bothers" the forum admins/users and develop solutions that solve these problems accordingly. You said that you've never operated a forum on the admin side. This needs to change right away because it's the only way you can identify the shortcomings of other forum software. Talking to the owners of different forums will also greatly help here.
I wish you good luck!
I really like MyBB. It's really easily extensible, fast and requires no special software (can run on standard €1 shared hosting). Those are IMO absolutely crucial features
We used SMF (Simple Machines Forum) for a long time but that is just, outdated at this moment with these new forums coming out. MyBB is a pretty good free version too!
About paid ones, absolutely vBulletin.
Regards, Armand
A very active forum used to be de facto symbol of a healthy website.
Well, used to be.
Just avoid Vbulletin and you will be fine. It's horrid now.
No there's no way I'd attempt anything like that. The reason I don't like most of the existing stuff is that it has too many features, not too few. My thing would be very light and simple compared to the stuff that's out there now.
Yes, that's the purpose of this thread
You're probably right: another forum program is the last thing the world needs.
Yeah but it is not about what you want, but what do users want. It doesn't matter if you and I don't like too many features or a bloated interface - you'll still receive requests like "on phpBB I could do that, and on Discourse, I can do this and that".
The most successful forums I know of (Reddit and Hacker News) are quite primitive in terms of features. That would be the approach I'd be after. At the end of the day, users who want phpBB or Discourse know where to find them.
Yeah I guess so. Maybe the cached format would be a "command list" of rendered html and dynamic insertions. That would at least get rid of parsing the user posts on every page view. Some variants of that are possible. In the extreme, the cached format could even be executable machine code from a JIT compiler.
Good point about gzip(A)+gzip(B) possibly not working the same way between gzip and zlib. It's unfortunate that gzip content encoding doesn't give a way to preload a compressor state the way zlib does. That's a standard trick for compressing lots of independent small chunks of text (preload the compressor with a bunch of text statistically similar to your chunks, save the state, then compress and uncompress each chunk starting from that state). Compressing like that might still be a good way to save some ram or db space on messages.
K.I.S.S. rules, I guess.
Keep It Simple Stupid.
K.I.S.F - Keep It Simple & Fast.
Yep. I've just been looking at the hacker news source code (http://ycombinator.com/arc/arc3.tar). It's written in Arc (somewhat bogus Lisp dialect) and is completely self-contained (uses no database, has its own web server, Markdown conversion, etc). The main app is about 2.6kloc of Arc and there's another 2.5kloc or so of support libs in Arc, plus the libs and compiler for Arc itself, about 5k total of Arc and Scheme).
It includes Reddit-like story and comment voting/flagging and ranking, an anti-procrastination field you can set in your profile (prevents you from viewing the site above X frequency), and a fair amount of anti-spam and anti-abuse stuff (throttling of new user posts etc). Pretty neat! I wouldn't use quite this implementation approach but it seems worth studying. It isn't bloaty but it's NOT ultra-concise code. A fair amount of it manually does stuff that's built into more modern languages or their standard libraries.
The HN site that it serves gets around 350k unique visitors a day and 608k views/day according to https://hypestat.com/info/news.ycombinator.com fwiw. It has no ads but that stats site claims it would get around $4k a day of ad revenue, which sounds extreme ($6/CPM even for that specialty tech audience?), but whatever.
I look at Discourse which is 230K lines of Ruby not including the monstrosity that is Rails plus the SQL database etc etc., and it makes me want to cry.
The first time I ran into Discourse forum, it was Eve online community forum. I liked it at first and then started to grow my dislike at it.
Like I said, it's over engineered.
Storing a parsed copy of the post is quite logical and already done on a number of forum systems out there. This is mostly useful for slower scripting languages (PHP) which are slow to traverse text. If you're writing in a faster language, caching a parsed copy might not yield much benefit. A C implementation of a parser can easily traverse text at >100MB/s on a modern CPU core, and recall that CPU cost is often not much of an issue for typical forums whereas I/O usually is.
A number of PHP templating systems actually generate native PHP code for what they do (PHP is a templating language after all). With a PHP opcode cache installed, this is fairly close to your idea for fast templates.
You still have the problem of specifying the dictionary, which there's no solution to there (if server side only, you've got a lot of control though). Also note that preset dictionaries can only prime the LZ77 state, not Huffman, though at ~1.5KB chunks, it may not matter all that much.
I haven't looked into it, but SDCH or Brotli may have preset dictionary capabilities, if you don't care about compatibility.
Maintaining a preset dictionary across posts is likely difficult. To be honest, it probably makes more sense to let compression be done by the DB rather than handle it at an application layer (if anything, admins will hate you if they can't edit stuff in the DB).
For zlib you can always use a faster zlib library - Cloudflare's zlib forked library is faster than standard zlib
I use it on my Centmin Mod Nginx zlib builds and optionally for PHP zlib library with Cloudflare zlib and HTTP compressed request benchmarks are at least 29% faster than standard zlib for HTTP gzip compression.
Not wanting to necro but having been on LET's mod team since last night, I can already see there are plenty of issues facing the software that hadn't sunk in for me as an ordinary user, so I thought I'd acknowledge that.
Also, I mostly lost interest in pursuing this as a personal project, but now it looks like I can possibly get paid for it (not all that much though). So I'm thinking about it again. But I'll probably pick something different.
admin, moderation stuff is alot to deal with
Old-school presentation (like reddit, craigslist etc), site should work with no javascript, and not too much Discourse-like browser wizardry even if JS is enabled.
This might win you some points with the Federal government accessibility auditors, but otherwise...why? It's 2018. What is the glory in making javascript-free sites?
Heck, I think a SPA might be more interesting than something phpbb-like.
FYI Reddit redesign is built on React.
old.reddit.com bypasses it and is much better.
I disagree. I've been using the Alpha/Redesign for the past 6 months (I've been one of the first users in fact) and while half a year ago it seemed like a complete garbage, it has come a long way.
In my opinion it's much better than the old design.
Maybe you are right in some sense of it having better functionality. It hurts my eyes so much that I shut it off right away and kept using the old version. I can't stand it. Sooner or later I will probably write my own client and only use the JSON interface. Of course like Twitter they will eventually shut the JSON interface off, to stop people from bypassing the ads.
IMO user -> moderator isn't that big of a difference. Administration is the key thing and often is a much bigger jump. You need to design for an admin rather than a user, since admins will be choosing the forum software, not users.
I can't stand Javascript heavy sites - noticeably laggy even on a 4GHz CPU (and a disaster on mobile), and you can never really tell what browser features will work (like will trying to open a link in a new tab send me to the same page with a # on the end?). Also because using NoScript blocks half the nonsense on the web (coin miners, anti-adblock detectors, browser exploits etc).
Never understood those folk. They should stick to lynx, the command line, a sandbox, a VM, or heavy sedation.