Help with WCG

[Markfw]

I see @biodoc has been doing great at WCG. I have multiple machines that say "downloading". First, how do I fix this ? Second, what on earth are you all doing to get such good PPD ? I have 13 machines trying to get units, and most are stuck trying to download.

 

[pututu]

On the question of PPD, just crunch the OPNG tasks. That's the gpu version of OPN cpu task. They run faster on newer gpu cards and gives the highest PPD. You may need to spam the server and create multiple instances to feed your fast gpu cards.

On the question of stuck download, run a script to update the network periodically and wait.....

 

[Markfw]

On the question of PPD, just crunch the OPNG tasks. That's the gpu version of OPN cpu task. They run faster on newer gpu cards and gives the highest PPD. You may need to spam the server and create multiple instances to feed your fast gpu cards.

On the question of stuck download, run a script to update the network periodically and wait.....

OK, what is the update the network script ?

 

[StefanR5R]

About "optimizing" PPD at WCG: The OpenPandemics project cannot issue as much work as contributors are able and willing to process. (Or maybe they could; but then they would be unable to post-process all the results they receive.) Therefore, ask yourself if trying hard to work for OpenPandemics aligns with your own goals in your Distributed Computing hobby.

Regarding stuck transfers: The client will retry the transfers eventually, but the retry period is presumably rather long. That's a heuristic approach with the goal to avoid to congest an already congested server link even more. If this is not just transient but persisting, and you don't have means to help the project fixing it, then the only help to that project is obviously that a part of the prospective contributors backs out and leaves, until they have means to improve the situation substantially.

 

[Markfw]

About "optimizing" PPD at WCG: The OpenPandemics project cannot issue as much work as contributors are able and willing to process. (Or maybe they could; but then they would be unable to post-process all the results they receive.) Therefore, ask yourself if trying hard to work for OpenPandemics aligns with your own goals in your Distributed Computing hobby.

Regarding stuck transfers: The client will retry the transfers eventually, but the retry period is presumably rather long. That's a heuristic approach with the goal to avoid to congest an already congested server link even more. If this is not just transient but persisting, and you don't have means to help the project fixing it, then the only help to that project is obviously that a part of the prospective contributors backs out and leaves, until they have means to improve the situation substantially.

So, because they do not have capacity, they want people to just quit.

Well, maybe that is what I should do, too bad IBM quit supporting this.

 

[biodoc]

Here's the latest from WCG on network issues. It sounds complicated to me anyway. https://www.worldcommunitygrid.org/forums/wcg/viewthread_thread,44311_offset,80#676758

Yeah I'm running GPU & CPU tasks which is why my PPD is abnormally high.

 

[StefanR5R]

So, because they do not have capacity, they want people to just quit.

Well, maybe that is what I should do, too bad IBM quit supporting this.

They sure don't want people to quit — what I really wanted to express is that if the transfer problems persist, then the workaround (for now) is that fewer people try to transfer, not that the same number of people try harder to transfer.

As for the OpenPandemics issue: That's a problem of the subproject itself, ever since they improved contributors' throughput dramatically by introduction of the GPU application version. For them, it should be a mostly comfortable "problem" to have: Enabling the contributors to perform 100% of the work (per day, per week…) swiftly. For WCG contributors overall, the unfortunate side effect is that credit becomes completely meaningless, unless if you look at per-subproject credit.

Also, my post may have sounded as if I implied that e.g. @biodoc was "optimizing" hard. Please don't take it that way; I suppose @biodoc simply has got GPU-equipped clients attached at WCG, rather than just CPU-only clients. (Edit: Ninja'd by himself.)

Edit 2:

Here's the latest from WCG on network issues. It sounds complicated to me anyway. https://www.worldcommunitygrid.org/forums/wcg/viewthread_thread,44311_offset,80#676758

Thanks for this pointer; interesting info.

 

[mmonnin03]

It would help if all of the input files were zipped. Instead of having 5-6 files per task and hammering connections for each file, just have the larger 1 file

 

[StefanR5R]

5-6 files per task

Ouch! Is this new, or inherited from IBM's WCG?

 

[mmonnin03]

I don't recall.
I haven't had any transfer issues when there has been only CPU work available. The GPU work completes so fast and causes the slow down for all tasks across the project.

 

[Kiska]

Ouch! Is this new, or inherited from IBM's WCG?

I believe this issue is inherited from IBM. However IBM had a more capable team to manage the issues

 

[mmonnin03]

Finally some of the stats on My Contribution are being updated.

Bunch of GPU work available today.

 

[StefanR5R]

… most of the web pages unavailable today…

"an error occurred while processing your request", or "org.hibernate.exception.GenericJDBCException: Unable to acquire JDBC Connection"