Changeset 2286 for trunk/src/Scantable.h

Timestamp:

09/02/11 19:05:11 (13 years ago)

Author:

Kana Sugimoto

Message:

New Development: No (performance tuning)

JIRA Issue: No

Ready for Test: Yes

Interface Changes: Yes

What Interface Changed: a parameter "filename" is added to Scantable::summary. scantable.summary doesn't return a string anymore

Test Programs: sdlist unittest/ scantable.summary("summary.txt")

Put in Release Notes: Yes

Module(s): sdlist, asap.summary

Description:

scantable.summary is very slow for large data sets (in row number) often outputted
by modern telescopes. It takes > 1.5 hours to list OTF raster scan with 350,000 rows.

This was because, the methods accumulates the whole text string (~700,000 lines) and
returns it as a string. Once the summary string exceed several tens thousands lines,
elapse time increases non-linearly, may be because very massive output string starts
to overweigh the memory.

I updated scantable.summary so that it flushes the summary string more often to file/logger.
After the modification, scantable.summary could list the data mentioned above in ~ 7 minutes.
The side effect of it is that scantable.summary doesn't return summary string anymore.
(But people may not happy with sub-million lines of string anyway.)

File:

• trunk/src/Scantable.h (modified) (1 diff)

trunk/src/Scantable.h

@@ -375 +375 @@
 
   std::string headerSummary();
-  std::string summary();
+  //  std::string summary();
+  void summary(const std::string& filename="");
   //std::string getTime(int whichrow=-1, bool showdate=true) const;
   std::string getTime(int whichrow=-1, bool showdate=true, casa::uInt prec=0) const;