Celebrating the first year of IPython logging

It was last year today when I first started logging my IPython sessions explicitly using Pierre Raybaut's idea. All you need to do is just to make the changes/additions described in this piece of documentation (Logging to a file) on its soon-to-be-change LP repository. (Applies to IPython 0.10 and below)

You get a time-stamped log file (in your ~/ipython/ or wherever your IPython home directory is set at) created per day that looks like below:

#!/usr/bin/env python
# 2009-04-16.py
# IPython automatic logging file
# 13:15
# =================================
d = loadtxt(file, skiprows=30)
plot([d[i][8:] for i in range(12)])
# =================================
# 14:08
# =================================
boxplot(d[:][8:])

As of writing this entry I count almost 300 separate logs and combining them into one file using this little script yields about 37.5 k-lines (including lots of multiple entries, time-stamps, empty comments, many copy-paste codes that I haven't actually typed in)

Besides having this combined file as a rough measure for myself there is another good use for it as triggered with this question: How to exit IPython properly? IPython internal history file forgets what was in the session if you accidentally or intentionally kill your IPython session without issuing an Exit at the exit :) That new combined history file comes to our help.

First we will append all the time-stamp logs into one file (rename it to "history" so that IPython can load it at the start-up) Then from iplib.py comment the readline.set_history_length(1000) line to prevent 1000 lines limit in your history file. Now I can access all my previous coding history from within IPython again no matter how I end my sessions. (Providing that I will stitch my logs periodically)

Lazy coding at its best!

It would be great if IPython could handle history lines more smartly to read multiple lines back properly. Who knows maybe an IPython super user has a solution for that laziness as well.

By the way anyone knows how to remove duplicate lines from a file without actually sorting it?

GVIM+IPython with Conque plug-in

Here is an alternative way to bridge (G)VIM and IPython applications. GVIM is my editor-of-choice and IPython is a great interactive Python interpreter. If you are a user of these two environments, this simple integration technique could greatly boost your programming / prototyping speeds.

First go to conque (Thanks to Nico Raffo for the plug-in and helping me to include IPython functionality in it) page and install the plug-in following the simple installation instructions. (In my Fedora 12 system, I pulled the latest tar.gz package from the download list section and extracted it under ~/.vim folder.) Next grab the latest conque_term_pylab.vim file from the same section and place under ~/.vim/plugin directory.

When you open a simple Python script you can easily launch an "IPython -pylab" instance by hitting F6 and execute the whole script content by F8, use F9 to send a visual selection. The IPython inside the buffer acts as a part of GVIM, so you can easily switch between buffers and copy/move text from/to buffers. You can modify the IPython switch, default window position, mapped keys editing conque_term_pylab.vim

The screenshot below shows the bridge in action:

Hint: Add the following line into your vimrc file to get equally spaced buffer windows independent of the main GVIM window size.

autocmd VimResized * wincmd =

Currently only %run magic of IPython is implemented. Here are some of the ideas that could be added to improve this IPython + VIM integration:

• Add %whos key-mapping
• Launch IPython -pylab on start-up automatically if a Python script is opened.
• If "run" command is sent before an IPy launched instantiate one automatically.
• Make sure only one instance of IPython is running!
• GUI integration, create GVIM menus, reading %whos values back into a separate window and changing values.

These last points could be well achieved by working on PIDA project. It is very possible to have an VIM + IPython powered IDE that is especially suited towards scientific use.

Scenes from under a microscope window

This photo is showing about 40 μm dry soda lime glass microspheres. They are perfectly spherical indeed. We use these particles to calibrate optical particle detectors e.g. Forward Scattering Spectrometer Probe (FSSP) The picture with less beads shows even tinier particles -- I will say sizes down to 5-10 μm from the same bottle. However these are the smallest sizes it gets from the microscope we used. It is not possible to detect by eye the sphericity of microspheres from the 2.1 μm bottle.

The microscope we used is a Nikon SMZ-U (Zoom 1:10) Unfortunately, our efforts were not successful in trying to show the more sphericity of Ammonium Sulfate {(NH₄)₂SO₄} particles than Salt (NaCl) crystals. We definitely need a powerful electron microscope to confirm this riddle. See the rest of the images at my Particles titled Picasa album.

PS: In one of the pictures the hands belong to Julia Beard. Without those hands it would take me a day to finish this work. Thanks Julia for your patience and tolerating my never-ending requests.

PPS: This posting has really nothing to do with Dream Theater's Scenes from a Memory album :)