Tag Archives: python

Talk about Stoq on FISL14

I just came back from FISL14 at Porto Alegre – Brasil, where I gave a talk about Stoq. You can check the video here:

http://hemingway.softwarelivre.org/fisl14/high/41f/sala41f-high-201307041758.ogg

I just want to say thank you to my employer, Async Open Source, for allowing and providing me a way to go to the event. It was really great! Hope I can go again next year and give an even better talk!

Yet another Vim tips post

I’ve been using vim as my texteditor/codeeditor/ide/etc for about 2 years now. Since then, I tried a lot of different plugins, configurations, and end up with a good set for my workflow. I decided to share here some of those, maybe someone will find them useful 😛

First, here’s a list of my most useful plugins and some configurations I’ve done to them:

CtrlP is by far my most useful plugin. I used to use Command-T, but it’s dependency on ruby was giving me lots of headaches. It searches for files, recursively from the work directory (the place where you opened (g)vim) in a fuzzy way. You call it’s search function and start typing the file pattern.

ctrlp.vim plugin

You can also use it to search for opened buffers in the same fuzzy way.  And here is some options I’ve put on my ~/.vimrc:

" This will flush CtrlP and remake it's cache
nmap <C-F7> :CtrlPClearCache<CR>
" This will open the CtrlP fuzzy search window for files
nmap <silent> <Leader><Space>f :CtrlP<CR>
" This will open the CtrlP fuzzy search window for buffers
nmap <silent> <Leader><Space>b :CtrlPBuffer<CR>
" This will open the CrtrlP fuzzy search window for tags
nmap <silent> <Leader><Space>t :CtrlPTag<CR>

Indexer makes it very easy to automatically generate (and maintain) tags for your projects, so you can jump to them using CTRL+], g+] or even CTRL+Click. All you have to do is install it and it’s dependencies (see the homepage) and do a simple configuration.

There is tree ways of configuring it. One is by using it as an extension to project.tar.gz plugin. Another way is by creating a .vimprj folder inside your project’s folder, so as soon as it finds that folder, it’ll understand that that’s a projects’ folder and create tags inside that .vimprj. The last one, the one I use, is to create a ~/.indexer_files file and specify a projects parent folder. That way is perfect for people who keeps all of their projects inside the same directory, and it supports nested directories. Here is my configuration as an example:

[PROJECTS_PARENT]
~/dev/
~/dev/external/
~/dev/stoq/
~/dev/icmc/
~/dev/proj/

By doing that, the tags will be generated/updated at a folder ~/.vim_indexer_tags.  It’s so simple that I never have to remember to generate/update tags for none of my projects, even one that I just created (since they will be on one of PROJECTS_PARENT directories).

That’s another very useful plugin for people who uses grep a lot. It grep for a pattern, and populate vim’s quickfix window with all the found patterns, so it’s easy to access them, so you can navigate through them using ]q (the same as :cnext)  and [q (the same as :cprev).

And obviously, you can setup some maps to make it easier to grep your pattern. Here is a piece of my ~/.vimrc:

" Files to skip
let Grep_Skip_Files='*.bak *~ *.pyc *.o *.obj'
" Directories to skip
let Grep_Skip_Dirs='.bzr .git .hg'
" The grep itself
nnoremap <silent> <Leader>gg :Grep<CR>
" Grep recursively
nnoremap <silent> <Leader>gr :Rgrep<CR>
" Grep buffer
nnoremap <silent> <Leader>gb :Bgrep<CR>

Those are just my must-have list. There are some other that I won’t detail here but I find useful too, like:

To finish, here is some general-use pieces of my ~/.vimrc:

" Dictionary and Spell Options
set spelllang=en,pt
set dictionary+=/usr/share/dict/words
 
" Jump to the last position when the file was last opened..
autocmd BufReadPost * if line("'"") > 1 && line("'"") <= line("$") | exe "normal! g'"" | endif
 
" Visual help to keep text on 80 columns
set textwidth=79
autocmd BufEnter * hi ColorColumn ctermbg=lightred ctermfg=white guibg=#592929
if version >= 703
  autocmd BufEnter * set cc=+1
else
  autocmd BufEnter * match ColorColumn /%80v.+/
endif
 
" Clear highlight when refreshing.
nnoremap <C-L> :nohls<CR><C-L>
inoremap <C-L> <C-O>:nohls<CR>
 
" Make Y compatible with D
nnoremap Y y$
 
" Make Shift+Insert paste from global clipboard
nnoremap <S-Insert> "+p
inoremap <S-Insert> <C-O>"+p
noremap! <S-Insert> <MiddleMouse>
 
" Continous visual indenting
vnoremap < <gv
vnoremap > >gv

Most of those tips/plugins were found googling, on vim tips wiki and vim scripts itself.

Suggestions for improving this post are very welcome! 🙂

Writing asynchronous Python code with Twisted using inlineCallbacks

A few weeks ago, I started using twisted to develop a plugin for synchronization between Stoq and Magento.

Twisted is a really great tool, and it makes it very easy to write asynchronous code. The only problem is that your code has a high probability of becoming Spaghetti code :P. And anyone who knows me knows that I’m crazy when it comes to code organization.

Well, those days, specially after a hint by a work mate, Johan Dahlin, I started to take a look on inlineCallbacks decorator. It’s just beautiful and solve all my problems :).

Since I found the documentation a lot hard to understand, and very few examples on the web, I decided to try to make one of my own. Hope you will enjoy!

To start, consider the following piece of code (using the classic Twisted way):

from twisted.internet import reactor
 
def on_failure(err):
    print "Error:", err
    reactor.stop()
 
def on_success(*args):
    print "Success. Shutting down"
    reactor.stop()
 
def print_file(file_):
    d = async_print_file(file_) # This will return a Deferred
 
    d.addCallback(on_success)
    d.addErrback(on_failure)
 
def get_file():
    d = async_get_file() # This will return a Deferred
 
    d.addCallback(print_file)
    d.addErrback(on_failure)
 
if __name__ == '__main__':
    get_file()
    reactor.run()

On this example, we want to get a file, print it, and then shutdown the application. Yeah, it’s ugly, a little spaghetti (could be a lot more if the code wasn’t a simple example)… but it works.

Just for a fast explanation, the async_*() functions are fictitious functions that will return a Deferred. When it’s fired, it’ll call the function added by addCallback, or, in case of failure, the one added by addErrback. If more than one callback (or errback) is added, when the first one returns, that return value will be passed to the second function, and so on, as a chain of callbacks.

Now, take a look at the following piece of code (utilizing the inlineCallbacks way):

from twisted.internet import defer, reactor
 
@defer.inlineCallbacks
def print_file():
    try:
        # async_get_file still returns a Deferred
        file_ = yield async_get_file() # After yield, it's not a Deferred anymore
        yield async_print_file(file_)
        print "Success."
    except Exception as err:
        print "Error", err
    finally:
        print "Shutting down"
        reactor.stop()
 
if __name__ == '__main__':
    print_file()
    reactor.run()

Beautiful, isn’t it? Well, not really the most beautiful piece of code you will see around the world, but better than the previous example. What does all the magic is the yield statement (without using it, file_ would still be a Deferred)

When the code inside a function decorated by the inlineCallbacks decorator yields a Deferred (in that case, a function that returns a Deferred), the code goes on and the reactor will come back after the Deferred fires. It’s return value will be returned on the yield statement, and, if any errors occurred, the exception will be raised (that’s why I yielded inside a try/except clause).

Note that, because yield is captured by inlineCallbacks, there’s no way to use that function as an iterator generator.

And if we need to call another function decorated by inlineCallbacks? How to get it’s return value, as the return statement won’t work? Well, that’s why there is a function called returnValue. Take a look at this piece of code:

from twisted.internet import defer, reactor
 
@defer.inlineCallbacks
def get_arg():
    retval = yield another_async_func()
    defer.returnValue(retval)
 
@defer.inlineCallbacks
def print_file():
    try:
        arg = yield get_arg()
    except Exception as err:
        arg = None
 
    try:
        file_ = yield async_get_file(arg)
        yield async_print_file(file_)
        print "Success."
    except Exception as err:
        print "Error", err
    finally:
        print "Shutting down"
        reactor.stop()
 
if __name__ == '__main__':
    print_file()
    reactor.run()

In this example, we assumed that async_get_file needed an expecific argument, that needs to be retrieved asynchronous too. By doing returnValue(arg), we make anyone who yields get_arg() to receive arg, or raise an exception if an error occour.

A little complicated but, after a while you get used to it! 😉

Any doughs?