Changes between Version 5 and Version 6 of ticket/370/ticket/370/TicketDetails/OpsReadFromObstore


Ignore:
Timestamp:
Aug 12, 2019 1:26:33 PM (4 months ago)
Author:
Jin Lee
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • ticket/370/ticket/370/TicketDetails/OpsReadFromObstore

    v5 v6  
    1313   * Estimate the amount of memory required to read observations and allocate space within OPS program. Then use PBS resource request just enough to finish processing. This is based on my hunch that the reason for the failure stems from the fact that there might be not enough observations in some PE's and so OPS is allocating memory to certain variables during CX creation, which may be empty (Again this is only my hunch).
    1414
    15 Another way is to let Ops_CreateODB read the obstore and then write out ODB1; then let Ops_ExtractAndProcess read ODB1 and then write back to ODB1. This method should be used as it produces updated ODB1 which can be used by VER. However it's hard to know whether OPS retrieved all the data from an obstore file correctly. To make sure that all data are retrieved use the first method and then put together the app config file using the second method while comparing the log output.
     15Another way is to let Ops_CreateODB read the obstore and then write out ODB1; then let Ops_ExtractAndProcess read ODB1 and then write back to ODB1. This method should be used as it produces updated ODB1 which can be used by VER. Here's a list of things to keep in mind when running Ops_ExtractAndProcess this way:
     16
     17   * make sure `maxbatchessubtype` is set to a high enough number to be able to read all the batches in a obstore file
     18
     19One difficulty when using the second method is that it's hard to know whether OPS retrieved all the data from an obstore file correctly. To make sure that all data are retrieved use the first method and then put together the app config file using the second method while comparing the log output.
    1620
    1721=== Resources ===