BOINCGames.com Final Four Sprint - Milkyway@Home

[Skillz]

The second sprint of the year has started. The project is Milkyway@home and uses CPU only now. These tasks are multi-threaded and I don't think their default 12 threads per task is the best option. I want to say 4 threads per task is the best setup, but you'll need to use an app_config to force that.

 

[StefanR5R]

The default thread count of the application depends on the number of available CPUs and can go up to 16.

The syntax of app_config.xml can be found in the MilkyWay message board, but here is a copy:

<app_config>
    <app_version>
        <app_name>milkyway_nbody</app_name>
        <plan_class>mt</plan_class>
        <avg_ncpus>4</avg_ncpus>
        <cmdline>--nthreads 4</cmdline>
    </app_version>
    <app_version>
        <app_name>milkyway_nbody_orbit_fitting</app_name>
        <plan_class>mt</plan_class>
        <avg_ncpus>4</avg_ncpus>
        <cmdline>--nthreads 4</cmdline>
    </app_version>
</app_config>

I still haven't measured which thread count is optimal, but I am convinced that anything less than 16 is better than 16. There are MilkyWay participants who run just 2 or 1 thread.

 

[StefanR5R]

If you haven't run MilkyWay on a host for some time (a couple of weeks maybe) or even never, this host will receive rather random credit at first. Good credit and bad credit. It should eventually converge to reasonable credit, but I am not sure how quick the convergence is. Perhaps not very quick if we look at it from the perspective of a 4 days short sprint.

 

[Skillz]

Thanks for the app_config. Can you confirm that I can make this change AFTER a task downloads and they'll switch to 4-threads a task or do I have to flush the cache and download a new batch?

edit
Seems you can make the change on the fly. I converted one of my Epycs to use this app_config and it went from 8 active tasks at 16C each to 32 active tasks (though they still show 16C on the "use" column.) CPU use went from ~70.0 to 130+ Perhaps I should limit how many run concurrently. 32 might be too many? I'll let it ride while I sleep and check it when I get up.

edit 2
Checked after I got out the shower. CPU use dropped to around ~110 so I guess the few 16C tasks that were running when I made the change was causing the higher CPU use at first. It's only a 64-core, 128 thread CPU. So 110 is well below the 128 "100% CPU use". Each individual task is claiming ~360% use so that checks out also. Might over commit the host if this still looks good when I get up.

 

[StefanR5R]

After you let the BOINC client reread config files, these changes will apply to all tasks which start or which resume from disk after this. boincmgr however will keep showing the old <avg_ncpus> value though, which is irritating. Furthermore, tasks which were started before you changed app_config and keep running throughout this change, as well as tasks which were suspended to RAM before the change and are resumed from RAM after the change, will not pick up the change.

[Whether suspended tasks are resumed from RAM or resumed from disk depends on whether or not "[ ] Leave non-GPU tasks in memory while suspended" is set. Which means that GPU tasks can only suspend to disk/ resume from disk, not from/to RAM.]

If you go as far as to shutdown and restart the BOINC client (or even the entire computer), then this means that the client will reread its config files and that all tasks will start or resume from disk subsequently. Therefore, a client shutdown/restart will apply these changes to all tasks. In addition, boincmgr will show the new <avg_ncpus> value after such a client restart.

Hence, no need to cancel already downloaded jobs if you want them to run with altered <avg_ncpus> or/and <cmdline>. This is true for all projects, not just MilkyWay.

 

[StefanR5R]

Might over commit the host if this still looks good when I get up.

A certain level of overcommitment is certainly called for if good host utilization is desired at MilkyWay. Some raise <ncpus> for this purpose. I am merely relying on Intel's HyperThreading and AMD's SMT as an overcommitment of sorts myself, combined with limiting the threadcount per task to way less than 16.

Would be interesting to make synthetic tests. But e.g. the sort of tests which I use to dial in my computers for PrimeGrid, which involves running multiple copies of one and the same workunit at once, might not be the ideal approach for MilkyWay machine utilization tests. Maybe a few different workunits should be built into such a test. (Edit: On second thought, there is the trouble that credit cannot be precisely normalized between different workunits. That is, PPD comparisons between different test configs would be apples to oranges.)

 

[Skillz]

The team is kicking butt on the Milkyway Sprint and @StefanR5R could beat all the other teams on his own as well! Extremely impressive!

 

[Skillz]

We are still looking really, really good on the MW Sprint. TAAT members contributing to the Sprint.

 

[TennesseeTony]

A LOOOOONNNNGGGG time ago, I was browsing the MW forum and came across a user complaining that the multi-threaded CPU tasks were not fully utilizing his 14 core server chip. My reaction (hopefully I didn't post anything, personal thoughts instead...) were much the same as some users react to cloud BOINCing. I mean, I have a top of the line Pentium 4 3.0 GHz with one core, and HT. And here is this ****ole with a SERVER! Not fair. LOL!!

Fast forward to today and it seems there has been no changes, MT CPU tasks still run single thread for a significant amount of time, then jump to multi-threaded and finish quickly thereafter.

 

[StefanR5R]

On my 128-threaded hosts (running 32 tasks at once, 4 threads each), the load averages are 118…122, and on 88-threaded hosts (22 tasks × 4 threads), it's 79…83. That is, if one caps the thread count per task,

1. more tasks run at once, which makes sure that there are always several tasks which are currently in their multithreaded stage
2. the multithreaded portion of each task takes longer, which shifts the overall average of "wall clock time" spent towards the multithread parts,
3. synchronization overhead within the multithreaded part is lower, which means more computation in each thread and less waiting for semaphores and the likes.

(One downside of optimizing your hosts like this is that the CreditNew algorithm needs time to figure out whether your host performs very well indeed, or the tasks given to it were just lighter ones.)

In physics simulations, it's pretty much inevitable that there are some singlethreaded parts in the program (model setup etc.). On the upside, in case of MilkyWay@Home's NBody application, it is at least not as bad as in certain other physics projects' applications, which even perform considerable network transfers within the science task instead of leaving this for the BOINC client to deal with separate from the science tasks.

 

[StefanR5R]

Half a day to go. A nervous wreck is what's left of xii5ku, as Planet 3D Now! as well as GPU Users Group have gotten closer and closer by now. =:-O

;-)

 

[Skillz]

Yeah, a little closer, but you're still beating them by yourself. lol

 

[Ken g6]

Yahoo! The last stats update went through this time! Also, I beat @Icecold.

 

[Skillz]

Yahoo! The last stats update went through this time! Also, I beat @Icecold.

I never doubted your programming skills! Dang good work my friend.

Also congrats to the rest of the team for the absolute domination on the MW sprint. Excellent work!

 

[Skillz]

Final standings of our teammates. Great work everyone!