New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
SQLite has its own limitation. A couple of days ago I tested to load around 50,000 articles 500 words in average, to a SQLite based CMS I made.
I export it in chunks, and after all are properly set, it took around 8 seconds to load a page. After tweaking, it only comes up to 5 seconds to load a page. The performance also drop when I integrate visitor count and IP logging as it needs too much writing to the database.
I am going to try to use the Pear Cache_Lite and see how it goes.
@vRozenSch00n : That looks way too slow. Something's not right. Try these pragmas :
If you have only one process writing to the SQL, add
locking_mode = EXCLUSIVE. Otherwise use transactions for all your writes - you could be wasting time acquiring and releasing locks. Tweakingmmap_sizemight also help. Set those pragmas when you create the file for the first time, and again while opening it. Did you runANALYZEandVACUUMafter populating your DB?EXPLAIN QUERY PLANhelps me a lot in figuring out how to optimize DB tables. For improving read speeds, index your tables appropriately and create views for common SQL queries. On the other hand, indexes can slow down write speeds. I use views for everything from JOINS to text processing. Here is an exampleCREATE VIEW my_json_map AS SELECT '"' || id || '": ' || val || ',' FROM summary;Use prepared statements for everything - you can cache your prepared statement even if it seems a little overkill..
EDIT: Experiment with the
synchronouspragma if it fits your use case. Check the documentation before you proceed.@joepie 's pricing is more than fair. When I did 1-on-1 training I charged more than double his rate. For @_rm and his village in Africa that is mad about price, maybe he should stop paying for his boxes too because cost is an insult the Sahara desert nomads - not to mention the North Korean population.
How to offer free trial without actually costing you time for each client - simple: Pick a small subject and make a 30 minute tutorial over it. Post it online and point your leads there. The prospective client will know if he likes your style or not and you won't have to waste time for each lead individually.
Good luck @joepie
It is very easy for front-end guys to move to the back-end.
The real reason for NodeJs 's traction though is that is was the first environment geared exclusively towards async IO at the right time. Sure there were Python libraries and others but nothing came close in terms of dedication/goal/commitment.
As it got traction, it got a huge ecosystem which makes it an easier choice for its niche based on (say) Rust or Go. Plus with ES7 it's really awesome to write async code. It's like upgrading from a bicycle to a car on that area compared to ES5.
Example:
I already kind of do that (and that particular article has been very well-received so far), but an article or tutorial is still very different from 1-on-1 tutoring. It only works to some degree.
It's worth noting that there are some concerns over async/await that are worth considering. It is definitely a loss of flexibility in some ways, and I'm not yet convinced that they should exist in JS, at least in their current form.
Good reasons.
Still not sure on the non-blocking/asynchronicity thing. Is this w.r.t. to it being a webserver? Anything that O_NONBLOCK & O_ASYNC can't cater to?
I guess it just boils down to less complexity = more productivity, for many cases.
I meant video.
I took a quick look at the article's points. It sounds more than bitching than anything concrete. Some edge cases don't fit your (his) style? Great, use promises/generators directly on those. The elegance of the async/await style is fantastic. It would be like throwing away a Veyron because you're having issues using it on your corn field. Instead, the sane solution is to use a tractor when a tractor is best suited.
That'd be a bad idea for the same reason I don't normally do audio/video calls for tutoring. Text is just the most effective and accurate way to get technical information across.
I recommend reading it entirely. It's a fairly in-depth and technical article.
All I/O in Node.js is asynchronous. Filesystem reads, network connections (whether to a HTTP client or a database server), and so on.
That's pretty much the selling point, yes
It's not that it isn't possible in other languages, it's just far more of a headache.
Exactly. And less monitors thrown at the wall when you spend hours to discover a subtle bug in your threaded code.
I did read it
I just didn't do it the slow way - instead I read his points but glossed over the code examples. But thanks for the link, I had not bumped into that article before.
Oh, wow, I hadn't realized that.
Or the CMS you wrote is not efficient.
Apparently yes
after tweaking several things as per @rincewind suggestion, strip down the coding, I found out that I made a boo boo in the database calling procedure.
Too many unnecessary iterations
, and I think I have to change the database structure.
I guess I have to rewrite the code
If you are dealing with text documents, look at the [Full Text Search (FTS)
(https://www.sqlite.org/fts3.html) extension. It provides MATCH which is way faster than a LIKE, but restricted - it only matches tokens/words.
I assume you are storing documents as table columns. If you store them compressed, you can still use FTS4. For a memory constrained LEB, you can skip compression and use FTS5 instead. FTS5 allows stemming tokens (MATCH 'correction' will also match 'corrected' and 'correcting').
It seems unlikely for your case, but if you have tall and thin tables with non-integer keys, then try the
WITHOUT ROWIDoptimization, described hereNo comment about the offer itself but I was recently told that I couldn't post a much cheaper offer for something slightly related because the price, despite being dirt cheap for what it is, would raise eyebrows.
Thanks @rincewind. Yes I'm dealing with text documents. Sorry I'm a little bit busy and have to go now, I'll let you know the result then.
@vRozenSch00n: Take it easy! You dont need to explain, or respond
For logging, I would suggest RRDTool, or HDF5 for more complex needs. SQLite is not meant for frequent writes, but will give you good read-performance.
Well, actually I like to discuss things like this and I appreciate every suggestion and ideas. I love to try things so, I guess we'll always be in touch