Changeset 2286 for trunk/src/python_Scantable.cpp

Timestamp:

09/02/11 19:05:11 (13 years ago)

Author:

Kana Sugimoto

Message:

New Development: No (performance tuning)

JIRA Issue: No

Ready for Test: Yes

Interface Changes: Yes

What Interface Changed: a parameter "filename" is added to Scantable::summary. scantable.summary doesn't return a string anymore

Test Programs: sdlist unittest/ scantable.summary("summary.txt")

Put in Release Notes: Yes

Module(s): sdlist, asap.summary

Description:

scantable.summary is very slow for large data sets (in row number) often outputted
by modern telescopes. It takes > 1.5 hours to list OTF raster scan with 350,000 rows.

This was because, the methods accumulates the whole text string (~700,000 lines) and
returns it as a string. Once the summary string exceed several tens thousands lines,
elapse time increases non-linearly, may be because very massive output string starts
to overweigh the memory.

I updated scantable.summary so that it flushes the summary string more often to file/logger.
After the modification, scantable.summary could list the data mentioned above in ~ 7 minutes.
The side effect of it is that scantable.summary doesn't return summary string anymore.
(But people may not happy with sub-million lines of string anyway.)

File:

• trunk/src/python_Scantable.cpp (modified) (1 diff)

trunk/src/python_Scantable.cpp

@@ -118 +118 @@
           boost::python::arg("unflag")=false) )
     .def("_save",  &ScantableWrapper::makePersistent)
-    .def("_summary",  &ScantableWrapper::summary)
+    //.def("_summary",  &ScantableWrapper::summary)
+    .def("_summary",  &ScantableWrapper::summary,
+         (boost::python::arg("filename")=""))
     .def("_list_header",  &ScantableWrapper::listHeader)
     //.def("_getrestfreqs",  &ScantableWrapper::getRestFrequencies)