Data Streams Problem

[naeemb123]

Hi,

I am wondering if it is possible to create 3 upstreams and 3 downstreams, and once they have been processed by the cores, prepare another set of 3 upstreams and 3 downstreams.

I am creating an image processing program which requires large data to be processed in parallel but the cores can only handle a certain amount of data. How can I send data back and forth to the cores until all data can be processed?

Is there a way of doing this ?

[Tombana]

Hi,
You can create as many streams as you want, as long as there is enough memory for them. From the Epiphany cores, you can call ebsp_host_sync. This will trigger a callback on the host side and in this callback you can read out any processed data and fill up the existing streams with new data.

[naeemb123]

Hi Tombana,

Thank you for your help!
One question, when calling the function ebsp_set_sync_callback() on the host, is this before the ebsp_spmd() or after ?

[naeemb123]

Also, after using 'ebsp_host_sync()' from the Epiphany cores, do you have to call ebsp_reset_down_cursor(0); to reset the cursor of the stream or does this happen automatically when you put new data into the existing stream?

[naeemb123]

When you said the following " This will trigger a callback on the host side and in this callback you can read out any processed data and fill up the existing streams with new data."

How do you refill the existing down stream with new data on the host side?
As far as the documentation goes, you can only set the data upon creating a down stream
e.g. ebsp_create_down_stream(DATA,.....);

Essentially I will be creating new streams rather than fill existing streams right ?

[Tombana]

• You have to call ebsp_set_sync_callback() before calling ebsp_spmd().
• Indeed, you have to call ebsp_reset_down_cursor(stream_id);. This does not happen automatically, thank you for noticing, we will put that into the documentation.
• You are right, the only way to add data is with ebsp_create_down_stream. This is however not scale-able because the memory will get full this way. The next version of ebsp fixes this. You can get the develop branch of the repository and then there will be a new function on the host side:
  void* ebsp_stream_create(int processor_id, int stream_size, int token_size, const void* initial_data);
  This function creates a stream and returns a pointer. Save this pointer, and at any time you can write data to this memory and the cores will be able to read it. This way you can use existing streams. The documentation for this new version is not yet online but it should all be available in the header files (include/e_bsp.h and include/host_bsp.h)

[naeemb123]

Thank You Tombana, Really appreciate the help.
I will experiment with the new function and give you feedback on this new feature.

Many Thanks,
Naeem

[naeemb123]

Just some questions for clarification:

1. When you create a generic stream e.g. ebsp_stream_create on the Host side, without placing any intitial data onto it. Its regarded as an upstream right ? And vice versa when you create a generic stream with data, then its regarded as a downstream ?

2. When you've created these streams assuming that the above question is correct. From the epiphany core side, will you open up streams, read from them, pass them back up to the Host, using the same functions as you would use before ? e.g. ebsp_open_up_stream(), ebsp_move_chunk_up, ebsp_open_down_stream(), ebsp_move_chunk_down() etc. Will these functions know which generic stream is an upstream and which stream is a down stream ?

[Tombana]

1. The new streams are both up and down at the same time. You can fill them with data when you create them on the host, but you do not have to. If you do not add data, it will simply have zero tokens in it.

I forgot to mention: the indices of the stream are now global indices, instead of indices PER core.

2. On the Epiphany side, you can use ebsp_stream_open and ebsp_stream_close to open and close streams. Then you can use ebsp_stream_move_down and ebsp_stream_move_up to move tokens from or into the streams.

With these new streams, the old functions (ebsp_move_chunk_down and so on) do not work properly anymore.

In include/e_bsp.h there is documentation for each function, given in the comments above the function.

[naeemb123]

Thank you.

Im not entirely sure what you meant by global indices. Will this change how I read data from a generic stream in the cores ?

[Tombana]

By global indices I meant that when an Epiphany core refers to a stream with stream_id = 3 then it is the 4th (zero-based indices) stream that was created on the host. Different cores can open the same stream if they want (though not at the same time). In the old system, stream_id = 3 meant that it was the 4th stream created for that specific processor, and then you could not access a stream that was made for another core. In the old system, the stream_id index was local for each core, and in the new system it is global, meaning shared by all cores.

[naeemb123]

So if I have different sets of data for each core,
And I pass data using a for loop and calling ebsp_stream_create(..) inside the for loop, specifying the processor id, and specifying the processors unique set of data,
When opening the stream from the cores, would all the streams contains the same id ?
or would each core have different stream id's ? e.g. core 0 = stream_id 0, core 1 = stream_id 1.. etc
to access their set of data

[naeemb123]

I tried using: ebsp_host_sync(), ebsp_stream_seek(..) after reading in all the chunks. but it gives me the following error:
'BSP ERROR: Stream 0 has 8 space left, token of size 80 can not be moved up.'

for loop twice{
for loop chunks{
for loop chunk_size/sizeof(..){

         ...
     }
         ..
}
    ebsp_host_sync();
   ebsp_stream_seek(1,INT_MIN); 

}

Also when using ebsp_stream_move_up();
when printing the data received from the epiphany cores on the host side. an extra two elements with values 0 are added in the first two elements of each chunk.

e.g. chunk_size = 16 * sizeof(float)
chunk = 6;
data = malloc(chunk*chunk_size);

//create generic stream for upstream
//create generic stream with data for downstream

//after ebsp_smpd();

//print upstream
upstream[core][0] = 0.0
upstream[core][1] = 0.0
upstream[core][2-18] = correct output
upstream[core][19] = 0.0
upstream[core][20] = 0.0

Not sure whether the data is being streamed back up correctly or not. I can attach my dummy test code here if it helps.

[naeemb123]

If it makes it easier Tombana, If you can attach a simple sample code to show how to refill existing streams with new data using the new functions from the new version, that would make it clear how well the new version works?

Im working on a project for the University Of Glasgow as my dissertation. I hope to complete this project using your library.

Best Regards

[Tombana]

When opening the stream from the cores, would all the streams contains the same id ? or would each core have different stream id's ? e.g. core 0 = stream_id 0, core 1 = stream_id 1.. etc to access their set of data

The second statement is correct. When you create 16 streams, the first core can use stream_id = 0, the second core can use stream_id = 1 and so on.

I tried using: ebsp_host_sync(), ebsp_stream_seek(..) after reading in all the chunks. but it gives me the following error: 'BSP ERROR: Stream 0 has 8 space left, token of size 80 can not be moved up.'

When creating the streams on the host, you have to make sure they are large enough to accomodate for all chunks you plan on moving up into the stream from the core.

Regarding the first two elements of every chunk: you are correct. The first two elements of every chunk are used internally by EBSP. You can ignore them. They were not supposed to be outputted to the host, this is a known bug in EBSP. Untill we fix the bug I would recommend simply ignoring the first two elements of every chunk.

[naeemb123]

"When creating the streams on the host, you have to make sure they are large enough to accomodate for all chunks you plan on moving up into the stream from the core." - Great, Thanks that error is fixed.

Im still not sure how to refill an existing stream:
I tried the following:

float **downstream = (float **)malloc(float *) *bsp_nprocs());

//loop to create generic down stream to send data to each core
for loop
downstream[coreID] = (float*)ebsp_stream_create(...CoreData);

(you previously mentioned the ebsp_stream_create() function returns a pointer to external memory you can rewrite to, so I tried the following)

//callback
ebsp_set_sync_callback(callback);

//callbackfunction
void callback() {
printf("I am running") <<-- This works
//write new data to the generic downstream memory for each core
for loop each core
for loop chunk*chunk_size/sizeof(float) times
downstream[core][i] = newDATA; <<--re writing data to external memory
}

But this gives the following error:
BSP ERROR: Stream contained token larger (1084227584) than maximum token size (64) for stream (truncated)

All I want to do is replace the old data with new values.
How would I achieve this?

Regards