Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Idlers for Science! (How to run BOINC)

TrikeLikeTrikeLike Member
edited August 31 in Tutorials

If you've got some useless idlers sitting around, wouldn't it be nice if you could have them contribute their resources towards big scientific computing projects? We all know that compute has gotten more expensive recently, and you can help keep it affordable for researchers by running BOINC, a client for executing parallel scientific workloads.

What kind of science does BOINC support? In short, the types of scientific computing that involves pleasingly parallel workloads: math searching, protein folding, astronomy, and climate science. It's helped research topics as diverse as modeling Arctic sea-ice loss, gravitational wave searching, and finding alien civilizations in the night sky. If helping that interests you, read on!

Installation

You can run BOINC on just about anything, although the stronger the hardware, the better. BOINC is CPU-intensive, and the value your computer can provide is largely dependent on these things, in this order:

  • CPU & GPU performance (FLOPs)
  • Spare RAM (most jobs are very RAM-light, almost all below ~200MB)
  • Disk space (you'll want at least 5-10GB for several days of pre-fetched work + results)
  • Network speed

First, go to BOINC's install website and either download their binary or install it from your favorite package manager. Once it's setup and running, I recommend you install Podman or Docker, since the vast majority of complex jobs require it (otherwise, your computer won't be able to run them).

Once installed, run systemctl start boinc.service or simply boincclient (if you don't use systemd, you should clone their service file to your preferred format, otherwise you'll lose their sandboxing settings). Read the log output and ensure everything is set up right & that BOINC can connect to Docker/Podman (if you want it to do so). Now is also a good time to customize BOINC's performance settings; you can find the User Manual here.

Getting Workloads

Now that the BOINC client daemon is up and running, it's time to get some science. While you can join projects manually via the BOINC Project's website, it's better to join their curated job list, Science United, which features only projects vetted by the BOINC team. It also lets you edit your workload settings more easily via their website, and auto-adds you to new projects in your fields of interest.

Create an account for Science United on their site and then login. Once you've done so, run boinccmd --acct_mgr attach "https://scienceunited.org" $EMAIL $PASSWORD", and then you should see it sync up. This command is the main way you control the daemon; its man page is here. You'll need to rerun this setup command across each server you deploy this to.

On the Science United website, you can set options such as what types of research you'd like to help with, and adjust research utilization preferences for your idler(s). When you're done, run boinccmd --acct_mgr sync and then wait for it to update, or just restart the daemon & wait about a day (it syncs automatically, so you don't need to do this every time).

Now sit back and watch those spare cycles be used for scientific number-crunching! It can take a while for job servers to decide which jobs to give to your client, so be patient. (Also, without Docker/Podman, you're unlikely to get jobs quickly.) Science United will send you an email about once a week with information as to your total hours contributed across your cluster, and you can login at any time to see a graph of your compute donated.

FAQs

  • "Where's that one project about hunting alien civilizations (SETI@Home)?"
    Unfortunately, it shut down several years ago after completing its work. Until they come back, you'll need to hunt for aliens the old-fashioned way, with a telescope pointed at the stars.

  • "What about Folding@Home?"
    They do great work! However, the vast majority of their jobs are GPU-only, which means they're useless for garden-variety VPSes. You'll likely never see work from them.

  • "Will this thrash my VPS/use all the CPU-time?"
    Highly unlikely with the default settings. I let BOINC use as much CPU as it likes with auto-shutdown at 80% CPU use of non-BOINC tasks and haven't once experienced any serious slowdowns on my server whatsoever. All my Tor relays also run BOINC and the two happily coexist in practice (when Tor is using resources, BOINC automatically isn't and vice versa). Consider running BOINC in a systemd slice if it bothers you.

  • "Will BOINC consume a lot of disk/network?"
    It can, but only up to its self-assigned quota (it states this in the daemon startup output). If this worries you, disable pre-storing jobs in the config file or on Science United. It also doesn't use a lot of network, but you can also reduce it/run it only at certain times as well in the settings.

  • "Is it private and secure?"
    By most definitions, yes. You don't need to send any personally identifying info, although do note that obviously BOINC and Science United will know your server's IP and some hardware info for picking jobs, all of which can be seen in the Science United portal and /var/lib/boinc/client_state.xml. BOINC is 100% open-source and so are all workloads. It's important to note that Docker/Podman is not a good security boundary; most BOINC jobs run with relatively tight permissions with bubblewrap, but if you are paranoid about container escapes I suggest turning Docker/Podman jobs off.


Hope some of you find this useful; I've contributed several hundred hours by running BOINC across many idlers this year alone. (Note that I am not affiliated with BOINC; I just like to support the project.) Thank you for supporting the computational science community!

Thanked by 1oloke

Comments

  • aphexaphex Member

    Is this really idling if it is using 80% CPU?

  • @aphex said:
    Is this really idling if it is using 80% CPU?

    The whole point is that the server isn't idle anymore; now, it's doing science when it would have been idle. When the server isn't idling (80% or more CPU is requested by a non-BOINC process), BOINC will detect that and shutdown until it's idle again. You can adjust as needed.

  • What if I don't believe in science? Is there a BOINC for flat earthers?

    Thanked by 2Mainfrezzer rpqu
  • LeviLevi Veteran

    @Obelous said:
    What if I don't believe in science? Is there a BOINC for flat earthers?

    Pagan-BOINC - for those who believe in sun, earth and wind.

  • maybe we need a LET BOINC Team :D

    Thanked by 2oloke TrikeLike
  • @Obelous said:
    What if I don't believe in science? Is there a BOINC for flat earthers?

    Second this.

    I would let my stuff only run tasks of certain criterias. For example, discovering if politicians are capable of intelligent thoughts

  • raindog308raindog308 Administrator, Veteran

    @Levi said: Pagan-BOINC - for those who believe in sun, earth and wind.

    I believe in science but I also believe the sun, earth, and wind exists. I guess I'm trans.

    What if you just like Earth, Wind, and Fire songs?

    @TrikeLike said: BOINC is CPU-intensive

    Best case your CPU usage will be nerfed by the provider. More likely, if you're burning CPU 24x7, you'll be asked to take your business elsewhere.

  • rm_rm_ IPv6 Advocate, Veteran

    Only do that on dedicated servers. On VPS most providers assume (hope!) that you do not use your CPU continuously, else this breaks the entire economy of them being able to offer hosting for reasonably cheap.

    You are not creating more good by "contributing to science" by overloading a mere VPS without any GPU, than you're creating more evil and suffering by being an asshole to your providers and your neighbors on the node.

  • @rm_ said:
    On VPS most providers assume (hope!) that you do not use your CPU continuously, else this breaks the entire economy of them being able to offer hosting for reasonably cheap.

    Obviously you shouldn't be running it the whole time. BOINC only runs when it gets a fitting job that it thinks it can complete in the idle time resources you have available; with the default settings, long unbroken compute runs are rare. You can easily make it as light or as heavy as you want, so the choice to be an ass to your shared-server neighbors is entirely up to you.

  • My idlers are too dumb to be doing science

  • aphexaphex Member

    @TrikeLike said: BOINC only runs when it gets a fitting job that it thinks it can complete in the idle time resources you have available

    I honestly don't know what I can accomplish with 1/10 of a shared thread vCore for a few hours

  • @barbarza said:
    My idlers are too dumb to be doing science

    Idler not dumb, idler strong. Idler hunt in group!

    Thanked by 1TrikeLike
  • @aphex said:
    I honestly don't know what I can accomplish with 1/10 of a shared thread vCore for a few hours

    There's a lot more dumb parallel problems than you'd expect, mostly with primes and brute-force searching math problems. Also, over a very long period of time, even a trickle of compute adds up. Multiply by many idlers, and you'd get a decent number of hours. Idlers strong together.

  • rm_rm_ IPv6 Advocate, Veteran

    @TrikeLike said: complete in the idle time resources you have available

    On VPS there is no idle time that belongs to you. It belongs to the provider and other customers.

  • defaultdefault Veteran

    How can one set a CPU limit for this? Is there a setting?

  • forestforest Member

    How does this differ from cryptocurrency mining from the provider's perspective? Isn't it hammering the vCPU at 100%?

    Thanked by 1rm_
  • rm_rm_ IPv6 Advocate, Veteran
    edited August 31

    @forest said: How does this differ from cryptocurrency mining from the provider's perspective?

    It doesn't. And given all the overheads, wouldn't be surprised if "80%" usage inside the VM ends up being like 130% on the host.

  • forestforest Member

    @rm_ said: And given all the overheads, wouldn't be surprised if "80%" usage inside the VM ends up being like 130% on the host.

    If BOINC is purely computational like y-cruncher, it shouldn't be causing a storm of context switches (and thus vmexits) so it could be one of the rare cases where there's almost no overhead. Still, 80% 24/7/365 is rarely within FUP.

  • @default said:
    How can one set a CPU limit for this? Is there a setting?

    Use a systemd slice, or set the option in Science United, or edit the config file, as defined in /var/lib/boinc

  • TrikeLikeTrikeLike Member
    edited August 31

    @forest said:
    How does this differ from cryptocurrency mining from the provider's perspective? Isn't it hammering the vCPU at 100%?

    It can be, but BOINC is purely-computational (almost all userland) so there's little context switching, and as discussed, you can change its limits quite easily. For people using the default config, BOINC is very low nice, CPU-limits itself, and suspends when other processes are running, so it's not truly 80% 24/7/365 OOtB, maybe more like 25% or even lower (as it idles for sometimes hours while waiting for appropriately-sized jobs; for instance, a client with tiny resources isn't getting some huge stateful climate simulation job, but will instead get something like a small branch of a massively parallel numerical problem). You'd have to willfully disable the default settings and remove all built-in CPU limits for it to peg the vCPU at 100% constantly.

  • DrNutellaDrNutella Member
    edited August 31

    I get points for folding. I’m in top 10
    What do you offer? Or this?

  • forestforest Member
    edited August 31

    @TrikeLike said: BOINC is very low nice

    I assume you mean high nice? Low nice means high priority. Even still, that only affects the scheduler in relation to other threads on your system, not other VMs. Perhaps you could write something that pauses BOINC if CPU steal exceeds 0.5% for a few seconds while under load. That way you could be sure that you aren't being a noisy neighbor.

  • rm_rm_ IPv6 Advocate, Veteran

    @TrikeLike said: BOINC is very low nice,

    That does not translate to anything on the host node. Low nice load inside KVM is still high load on the host. Don't.

  • forestforest Member
    edited 12:19AM

    @rm_ said:

    @TrikeLike said: BOINC is very low nice,

    That does not translate to anything on the host node. Low nice load inside KVM is still high load on the host. Don't.

    It would be interesting if a VM could request that the host lower its priority. Something like SCHED_IDLE but for guests on the host such that the host would only consider the VM for scheduling if no other VM can be.

    Otherwise the best you can do is reactively monitor CPU steal to determine if you're interfering with others.

Sign In or Register to comment.