GVIM+IPython with Conque plug-in

Here is an alternative way to bridge (G)VIM and IPython applications. GVIM is my editor-of-choice and IPython is a great interactive Python interpreter. If you are a user of these two environments, this simple integration technique could greatly boost your programming / prototyping speeds.

First go to conque (Thanks to Nico Raffo for the plug-in and helping me to include IPython functionality in it) page and install the plug-in following the simple installation instructions. (In my Fedora 12 system, I pulled the latest tar.gz package from the download list section and extracted it under ~/.vim folder.) Next grab the latest conque_term_pylab.vim file from the same section and place under ~/.vim/plugin directory.

When you open a simple Python script you can easily launch an "IPython -pylab" instance by hitting F6 and execute the whole script content by F8, use F9 to send a visual selection. The IPython inside the buffer acts as a part of GVIM, so you can easily switch between buffers and copy/move text from/to buffers. You can modify the IPython switch, default window position, mapped keys editing conque_term_pylab.vim

The screenshot below shows the bridge in action:

Hint: Add the following line into your vimrc file to get equally spaced buffer windows independent of the main GVIM window size.

autocmd VimResized * wincmd =

Currently only %run magic of IPython is implemented. Here are some of the ideas that could be added to improve this IPython + VIM integration:

• Add %whos key-mapping
• Launch IPython -pylab on start-up automatically if a Python script is opened.
• If "run" command is sent before an IPy launched instantiate one automatically.
• Make sure only one instance of IPython is running!
• GUI integration, create GVIM menus, reading %whos values back into a separate window and changing values.

These last points could be well achieved by working on PIDA project. It is very possible to have an VIM + IPython powered IDE that is especially suited towards scientific use.

Scenes from under a microscope window

This photo is showing about 40 μm dry soda lime glass microspheres. They are perfectly spherical indeed. We use these particles to calibrate optical particle detectors e.g. Forward Scattering Spectrometer Probe (FSSP) The picture with less beads shows even tinier particles -- I will say sizes down to 5-10 μm from the same bottle. However these are the smallest sizes it gets from the microscope we used. It is not possible to detect by eye the sphericity of microspheres from the 2.1 μm bottle.

The microscope we used is a Nikon SMZ-U (Zoom 1:10) Unfortunately, our efforts were not successful in trying to show the more sphericity of Ammonium Sulfate {(NH₄)₂SO₄} particles than Salt (NaCl) crystals. We definitely need a powerful electron microscope to confirm this riddle. See the rest of the images at my Particles titled Picasa album.

PS: In one of the pictures the hands belong to Julia Beard. Without those hands it would take me a day to finish this work. Thanks Julia for your patience and tolerating my never-ending requests.

PPS: This posting has really nothing to do with Dream Theater's Scenes from a Memory album :)

Where you can't code...

Please don't have the feeling that a research aircraft isn't one of the places where you can't code comfortably :) This blog is more to do with post-philosophical investigations of the title.

This is one of the times that my programming literacy is to no avail to perform the research task ahead of me.

The problem is simply defined: Find the time-ranges where the research aircraft was sampling at cloud-bases.

What about the data-set?
About 37 hours of airborne data-set from the Saudi Spring 09 atmospheric measurement campaign. It contains aerosol and cloud micro-physical data and atmospheric state parameters from at least 10 different probes, listing over 100 different measured variables (e.g. pressure altitude, cloud condensation nuclei concentration, 2D images of ice-cloud particles.) We also recorded some of the flights from take-off to landing for nostalgia purposes and mainly aiding us while performing post-flight studies.

I get these ideas half-way through my manual exploration of the data-set. Many measurements are helpful in this analysis (e.g. air-dew point temperatures to determine where clouds are forming, the state of the aircraft in a given time-series plot, amount of liquid water content to distinguish in and out of cloud conditions, and most valuably in my opinion is visual observation from the recorded videos) helping me to infer when actually the aircraft was sampling right underneath a cloud in a level path.

This is part of my job and at the core of my thesis work. Even tough I complain little about the situation I get paid for what I am doing right now. My complaints are mostly for, in spite of all that rich data-set I am the one eventually making that final informative decision after manually and cautiously going through the data at hand. Far from being generic or universal. Good luck to myself if I need to extend the analysis for another airborne data-set :) I wish I had taken much wiser notes instead of trying to spot the most interesting occurring cloud of the day.

To complicate the analysis to a bit further level: not only find the cloud-base measurements also find the consecutive vertical passes of that same cloud in an automated fashion. No, not the one on its left nor the one on the right.

I am counting months backward for my graduation. Probably I won't have much time to see a breakthrough in AI research till then as mentioned in this article: How Long Till Human-Level AI?

Simple animation of UWyo CCN counter chamber

After modifying my Random particles in a cube example with Mayavi's @animate decorator I can get this simple animation which I use to demonstrate particle growth inside the device chamber. 

The Chemistry department of our school hosting a two-days Air Pollution Workshop for the neighbourhood high-school students. As a part of "Aerosol Particles" lab I am giving a demonstration about "Cloud Condensation Nuclei (CCN)" and University of Wyoming designed CCN counter.

The example is really the simplest it gets since it is designed for high-school students to little better appreciate the particle growth in a sealed chamber. There are many improvements could be made on this animation to make things much closer to what they are in reality. Such as:

• Particle locations should be depicted in a cylindrical space inside of a cube as given in this example. 
• Animation could be run in parallel with the counter by simply reading the status of the device through its serial port. (i.e., reading TWAIT, FLUSH, CCN_DETC status)
• A GUI could be added to control parameters of the instrument. (e.g., setting its supersaturation)
• Hot top-plate and cold bottom-plate could be drawn on newly created cylindrical chamber and supersaturation could be modelled and shown.
• Different continuous distributions could be added. (For now the particles are gamma distributed -using NumPy's gamma distribution function.)
• Instead of growing all the particles activation could be modeled more realistically (i.e. simply via κ-Köhler theory.) and applying condensational growth equation.
• Particle interractions could be included (e.g. gravitational settlings, collisions in between particles etc...)
• Particle detection could be modelled employing the Mie scattering theory.

This list could be expanded with many other considerations. Some are easy to implement, some really takes a while to achieve. Before I move further it is a good idea to read Dr. Snider's Supersaturation in the Wyoming CCN Instrument article while I have chance to spend time with the counter. In case you are curious about the code just follow the ccnanim.py link. Yes, all this buzz have been 33 lines of code. That's what you get when start coding with Python.

*

The bottom left photo is courtesy of Dr. David Delene (The photo should have been taken in one of Mali precipitation enhancement campaigns.) The bottom right one is courtesy of Dr. Manfred Wendisch in his Airborne Physical Measurements: Methods and Instruments lecture notes.

Py4Science @ University of North Dakota

On my second attempt to popularize Python and its scientific computing ecosystem I volunteered to introduce interested people in an interactive tutorial session as a part of the University of North Dakota 2010 Scholarly Forum program. The Graduate School kindly helped and supported me for this one-of-a-kind interactive presentation in the Forum history.

The session was advertised on the Scholarly Forum web-page and e-mails were circulated throughout the campus spreading the session information. The first half-hour of the tutorial was spent installing and demonstrating the PythonXY, Enthought Python Distribution [EPD] and Sage via the notebook interface. After introducing the Python language we briefly went over the most fundamental members of the Py4Science family; IPython, NumPy, SciPy, Matplotlib, SymPy, and Enthought Tool Suite [ETS]. Along with the technical demonstrations I presented how developments are made in open-source environments and try to emphasize the blurring line of user-developer distinction in Py4Science habitat. I provided some selected resources for the attenders to their further study. I finished the tutorial by showing some advanced level examples demonstrating the powers of Python in scientific computing. Lastly, I invited everyone to the upcoming SciPy10 conference that will take place in Austin, Texas starting on June 28.

You can access the slides of the tutorial by clicking this link.


Acknowledgements:

Although the original title of the tutorial was "Python and Scientific Computing in Open-Source" I have chosen to use Py4Science name by following the Fernando Pérez's tradition. I am thankful to my faculty advisor David Delene for joining us and introducing me and telling us an exciting Python news from the European Aerosol-Cloud Research [EUCARI] group. Susan Caraher of the Graduate School helped me with some of the logistics of the session and Vicki Thompson of the Continuing Education department provided eight PythonXY installed laptops for our use. Thanks to Enthought for allowing us to use multiple copies of their EPD. Finally, I am indebted to Jarrod Millman for providing me some of the t-shirts from the previous SciPy conferences. His kind jest inevitably doubled the joy of my teaching experience.

A Comparison of Measured and Predicted Cloud Condensation Nuclei (CCN) Concentrations

I presented my research on Tuesday at the University of North Dakota 2010 Scholarly Forum. Although a lot has changed from the original abstract I still provide it here for the curious. (Follow the links to access the PDF slides and the code and data.) Once I achieve the closure on the lab data I should be able to easily extend the similar methodology onto our field data.


Abstract:

    Research on CCN helps us to understand cloud and precipitation formation in a local scale and resolving the indirect aerosol effects on a global perspective. Following the conclusions of similar studies which emphasize the particle size distribution's being the biggest impactor on cloud nucleation ability of aerosols, and accounting for the chemical composition with the hygroscopicity parameter in κ-Köhler theory, a CCN closure study was performed. Measurements of the aerosol size distributions in the range of 0.1 to 3 μm is made by using a Passive Cavity Aerosol Spectrometer Probe (PCASP-100X), and the missing 0.01 to 0.1 μm range data is extrapolated with a spline interpolation technique. A two-parameter log-normal distribution is also estimated by fitting on the aerosol measurements including total aerosol particle concentrations measured by a Condensation Particle Counter (CPC - TSI-3771). Determination of the critical activation diameters is done by utilizing the κ-Köhler theory as proposed by Petters and Kreidenweis (2007). The closure resulted with the best R²=0.33 between the predicted and measured CCN concentrations at the 0.2% supersaturation (SS). Additional to this poor agreement, the concentrations obtained from the calculations under-predict the CCN concentrations over the full range of supersaturation measurements. The ground and airborne data from the spring 2009 Saudi Arabia field campaign are currently being analyzed.

PCASP aerosol number-size distribution

Following the similar approach given in Chapter 8 "Properties of the Atmospheric Aerosol" [Seinfeld, J. H., and S. N. Pandis, 2006: Properties of Atmospheric Aerosol, in: Atmospheric Chemistry and Physics: From Air Pollution to Climate Change. 2nd ed. Wiley-Interscience] one can construct PCASP aerosol number-size distributions as shown in the figure.

 

The code used for this demonstration is given at pcasp_aerosol.py