Help for a Jacket Newbie

[Old posts from the commercial version of ArrayFire] Discussion of ArrayFire using CUDA or OpenCL.

Moderator: pavanky

Help for a Jacket Newbie

Postby melonakos » Tue Jan 04, 2011 7:29 pm

One new Jacket programmer recently emailed the following:

I do have a further question. In my Matlab code, I need to perform both matrix operations and scalar operations. Initially I tried to put all variables on GPU, but found that GPU is extremely inefficient to perform scalar based computation, as I described in the post. Another way is to put the matrix computations on GPU and scalar computations on CPU. However, the communication between GPU and CPU (for example, using double and gdouble to transfer the variables) is also time consuming.

I have not got a method to solve this problem. Could you take some time to see my code to give some suggestions? Thanks!

This email contains two important truths:

1) It is not a good idea to throw all variables over to the GPU, because GPUs are not good at serial or scalar computations. It is much better to use the GPU in a "heterogeneous computing" fashion, wherein the GPU is utilized to do heavy lifting on the slowest parts of the code. In generate, this heavy lifting can be done effectively in two ways: a) by exploiting the data-parallelism of larger matrix operations, or b) by using GFOR to batch many smaller matrix operations into one big data-parallel operation. Code vectorization is often a very effective way of milking the most data-parallelism out of your code. (see

2) Data transfers between the CPU and the GPU should be minimized as much as possible. Everytime you send something to the GPU, you must make sure that the computations which will be performed on that data result in enough of a computational gain to compensate for the hit you took in the data transfer. The data transfer is on the order of milli-seconds, so if the computation is a micro-second computation that is bad and if the computation is a second or more, then you will probably find benefit in the GPU.

There are some guidelines on the wiki that we have developed to help new people work through understanding these concepts. We recommend the following links:

Hopefully these tips will help you out. You may try posting your code in these forums to see if you get any further help.

Good luck!

John Melonakos (
User avatar
Posts: 503
Joined: Tue Jun 10, 2008 9:49 am

Return to [archive-commercial] Programming & Development with ArrayFire