Parallelizing Bootstrapping At Least…

I understand that there are some workloads that don’t lend themselves to parallel processing. Yet bootstrapping is so clearly parallelizable. Why in this day and age, do we have to rely on serial processing of bootstrapping tasks, I don’t understand.

This became infuriatingly obvious to me when a bootstrapping task I left running on Friday was still running on Monday. Apparently somebody else was also suffering from this and took things in to his own hands.

I would like to thank Johannes for re-writing the irf function of panelvar package to utilize parallel processing.

https://github.com/hannes101/ParallelPanelVAR

The code and the examples are pretty clear. The approach can guide other work to also use similar solutions (NLSLoop… I am looking at you). One thing to take into account in this case is that free cores by themselves are not enough. Each process also consumes some memory to go along with it.

Finally the cores are not sitting idle while you wait.
Also consider the memory implications…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.