memory usage with gfor

[Old posts from the commercial version of ArrayFire] Discussion of ArrayFire using CUDA or OpenCL.

Moderator: pavanky

memory usage with gfor

Postby thomvil » Mon Dec 23, 2013 7:44 am

In reference to the memory considerations with using a gfor-loop (see, it is suggested to break up the loop into segments.

Code: Select all
gfor (array k, 400) {
  array B = A(span,k);
  C(span,span,k) = B * B.T();  // outer product expansion runs out of memory
for (int kk = 0; kk < 400; kk += 100) {
  gfor (array k, kk, kk+99) { // four batches of 100
         array B = A(span,k);
         C(span,span,k) = B * B.T(); // now several smaller problems fit in card memory

Are there some general guidelines in how big the segments can be. Or does this have to be done by trial and error?

More specific, in the example the loop of 400 is broken up into 4 batches of 100. I assume the increment (or size of batch) of 100 is dependant on the size of array B and device memory. Is there some way to deduce from this information that 100 is the ideal (or an acceptable) batch size?

Thank you
Posts: 2
Joined: Sat Dec 21, 2013 9:44 am

Re: memory usage with gfor

Postby pavanky » Tue Dec 24, 2013 2:03 pm


The ideal batch size will usually depend on the memory limits of the GPU you are running on.

You may want to fix the memory size the outer product can take and work backwards from there. I am not sure how much memory the rest of the variables take up. But if you know you have atleast 500 MB space for the gfor operation, you will be doing something like this.

Code: Select all
size_t maxmem = 500 * (1 << 20); // 500 MB
size_t mem_per_iter = A.dims(0) * A.dims(0) * sizeof(float); // Change float to double or cfloat or cdouble depending on the type
unsigned batch = maxmem / mem_per_iter;

Now you can use batch instead of 100 which would be ideal for your GPU.
Pavan Yalamanchili,
~ If it is not broken, you have not tried hard enough ~
User avatar
Site Admin
Posts: 1123
Joined: Mon Mar 15, 2010 7:39 pm
Location: Atlanta, GA

Return to [archive-commercial] Programming & Development with ArrayFire