Tags: 10-100, analyzes, application, arrays, cluster, functional, global, labview, program, programming, quotbuildingquot, returns, running

"Building" large arrays using functional global

On Programmer » LabVIEW

4,971 words with 5 Comments; publish: Thu, 01 May 2008 01:41:00 GMT; (20078.13, « »)

I have an application which analyzes data and returns a "results" cluster at

10-100 Hz. This program could be running from minutes to days a

nd I'd like to have access to all of the results clusters meaning if I store

them in an array it could

have something on the order of a million elements. I've written a func

tional global subvi that preallocates an array for the results and resizes i

t when the array becomes filled (doubling the size each time). This av

oids both having to allocat

e a huge array to begin with and using the "build array" vi too often.

This works but still doesn't seem that fast to me. In my actual appli

cation I have clusters with on the order of 100 elements and even when my ar

ray is only a couple thousa

nd elements big the profiler is showing that this vi is taking a few millise

conds each time. I've attached a sample version of the subvi and its b

lock diagram below. Are there things I can do to make this faster?&nbs

p; Thanks,Paul<img src="/app/links/?src=htt

p://forums.ni.com/attachments/ni/170/293644/2/Resizing%20array%20%28cluster%

29_bd0.jpg"> Message Edited by Phamton on 01-08-2008 01:36 PM

Resizing array (cluster).vi:

http://forums.ni.com/attachments/ni...3644/1/Resizing array (cluster).vi

Resizing array (cluster)_bd0.jpg:

http://forums.ni.com/attachments/ni...3644/2/Resizing array (cluster)_bd

0.jpg

All Comments

Leave a comment...

  • 5 Comments
    • Hello Paul, one easy mod is to physically move all constants in your diagram

      that are inside the while loop to outside the while loop, and of course con

      nect them back to necessary inputs inside the loop. I think that's a speed u

      p hint I recall from past d

      iscussions.

      Good Luck - Brian

      #1; Thu, 01 May 2008 01:42:00 GMT
    • First of all, nice work on the functional global.You might try separating yo

      ur Get and Set operations. Currently, everytime you Set an item into storage

      , you also Get a copy of the whole storage. I'd guess it's this extra copy t

      hat's hurting performance.I

      f you are setting multiple packets, but only getting the results for analysi

      s every once in a while, this should have a big effect on performance. Check

      out the two pics below. Notice that in the Set case, you're only outputting

      an empty array, and not ma

      king a complete copy of your internal data.tachments/ni/170/293653/1/Get.PNG"> ts/ni/170/293653/2/Set.PNG"> Message Edited by Jarrod S. on 01-08-2008 02:0

      2 PM

      Get.PNG:

      http://forums.ni.com/attachments/ni...93653/1/Get.PNG

      Set.PNG:

      http://forums.ni.com/attachments/ni...93653/2/Set.PNG

      #2; Thu, 01 May 2008 01:43:00 GMT
    • Couple of thoughts...

      When I last analyzed the time involved in building an array (LV 5.1?), I can

      to the conclusion that LV started out with about 1K by default. As long as

      the buffer stayed below 1K, I did not see delays trying to build. If I conti

      nued to build I saw another

      hit at 2K, 4K, etc. So... I think all of your work is just duplicating what

      the build array in a loop does.

      Another approach (remember this is coming from Ben, Mr AE) is just use multi

      ple queues. One for the GUI, one for logging, one for the analysis. The GUI

      and Logging queues you keep reading as normal. THe analysis queue just gets

      read once at analysis time.

      Another thought...

      How about using the in-place operations. They let you work exclusively in-pl

      ace.

      Yet another thought...

      How about pre-allocating 5X what you need? THen just take the sub-set when i

      t comes time to read.

      Final idea:

      Queues will often out perform AE since they can work in-place.

      Ben

      #3; Thu, 01 May 2008 01:44:00 GMT
    • "...from minutes to seconds and my code is way simpler;..."

      See <a href="/app/links/?link=http://forums.ni.com/ni/board/message?board.id=170&view=by_

      date_ascending&message.id=265705#M265705" target="_blank">here</a> for i

      n-place.

      So are we done here?

      Ben

      #4; Thu, 01 May 2008 01:45:00 GMT
    • Phamton wrote:

      I hadn't noticed the nifty option of "Return elements?" in the Get Queue Sta

      tus vi.

      Yes, I always forget about that as well, or I would have mentioned it. :smil

      eyvery-happy:

      from minutes to seconds

      I have to admit I didn't expect such an improvement when I originally sugges

      ted this, because your global was already relatively good, but I guess it de

      pends on the specific implementation.

      Note that queues can operate in-place, but they might still create copies, d

      epending on what you do.

      #5; Thu, 01 May 2008 01:46:00 GMT