Changes between Version 11 and Version 12 of Processing/lagdevelopersfaq


Ignore:
Timestamp:
Jul 6, 2012 5:45:32 PM (7 years ago)
Author:
jaho
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Processing/lagdevelopersfaq

    v11 v12  
    4444== 1. General [=#q1] ==
    4545
     46
    4647=== 1.1 What is LAG? [=#q1.1] ===
    4748
     
    7778
    7879== 2. Laslib [=#q2] ==
     80
    7981
    8082=== 2.1 What is Laslib [=#q2.1]
     
    204206
    205207== 3. Lidar Quadtreee [=#q3] ==
     208
    206209
    207210=== 3.1 How does Quadtree work? [=#q3.1] ===
     
    236239=== 3.4 How does caching work? [=#q3.4] ===
    237240
    238 Upon creation of the quadtree the user specifies a maximum number of points to be held in cache.
    239 
    240 
    241 
     241Upon creation of the quadtree the user specifies a maximum number of points to be held in memory. Whenever a new bucket is created inside {{{PointBucket::cache()}}} the {{{CacheMinder::updateCache(int requestSize, PointBucket* pointBucket, bool force)}}} method is called with {{{requestSize}}} parameter equal to the number of points that will be held in that bucket. This number is then added to {{{CacheMinder::cacheUsed}}} variable and compared to {{{CacheMinder::totalCache}}} equal to maximum number of points that can be held in memory. If the sum of the to is smaller then maxim cache size the {{{cachUsed}}} variable is updated and a pointer to the bucket is added to the queue ({{{std::deque}}}) of cached buckets. If total amount of cache requested is greater then allowed maximum the buckets are taken from the front of the queue and uncached until there's enough space for the new bucket. (A note here: the {{{force}}} variable in {{{updateCache()}}} method is actually obsolete as it's always set to true). \\ \\
     242When a bucket needs to be uncached, because there is no more room for new buckets in memory, {{{PointBucket::uncache()}}} method is called which serializes the bucket by compressing it and writes it to a file. Two lso compression is used for this together with two static variables to reduce the memory overhead. The serialization code looks like this:
     243{{{
     244#!div style="font-size: 100%"
     245  {{{#!c++
     246  // static shared working memory
     247  unsigned char *PointBucket::workingMemory = (unsigned char*) malloc(LZO1B_MEM_COMPRESS);
     248  unsigned char *PointBucket::compressedData = (unsigned char*) malloc(ceil(sizeof(LidarPoint) * capacity_ * 1.08));
     249
     250  // then inside PointBucket::uncache()
     251  lzo_init();
     252  lzo1b_2_compress((const unsigned char*) points_[k], sizeof (LidarPoint) * numberOfPoints_[k],
     253                             compressedData, &compressedDataSize_[k], workingMemory);
     254
     255  fwrite(compressedData, 1, compressedDataSize_[k], file);
     256  fclose(file);
     257  }}}
     258}}}
     259Once the data has been written, the bucket is deleted from memory and {{{CacheMinder::update_cache()}}} method gets called with negative {{{requestSize}}} to update the amount of used cache. \\ \\
     260If a {{{setPoint()}}} or {{{getPoint()}}} method is called on an uncached bucked it is simply re-created from the file an {{{CacheMinder::updateCache()}}} is called to reflect these changes.
    242261
    243262=== 3.5 What is the reason behind data compression? [=#q3.5] ===
     
    254273
    255274== 4. LAG [=#q4] ==
     275
    256276
    257277=== 4.1 How is LAG's code structured? [=#q4.1] ===