Rate Limiting Function Calls in Python with a Decorator

source: Stack Overflow “What’s a good rate limiting algorithm?”

I making this into a post because I have found it so handy. Some web APIs have rate limits on requests per minute or you may want to play nice and not overwhelm the service. In Python, you can use this decorator to rate limit a function that may handle the API access:

import time

def RateLimited(maxPerSecond):
    minInterval = 1.0 / float(maxPerSecond)
    def decorate(func):
        lastTimeCalled = [0.0]
        def rateLimitedFunction(*args,**kargs):
            elapsed = time.clock() - lastTimeCalled[0]
            leftToWait = minInterval - elapsed
            if leftToWait>0:
                time.sleep(leftToWait)
            ret = func(*args,**kargs)
            lastTimeCalled[0] = time.clock()
            return ret
        return rateLimitedFunction
    return decorate

@RateLimited(2)  # 2 per second at most
def PrintNumber(num):
    print num

if __name__ == "__main__":
    print "This should print 1,2,3... at about 2 per second."
    for i in range(1,100):
        PrintNumber(i)

This answer is simpler than setting up a queue system and is blocking, which can be good for sequential jobs.

Exporting from Aperture 3.2 to Flickr with location information metadata

Over the month of October, I took an incredible trip to Singapore and Japan. That trip deserves its own post, but until then, these photos will have to suffice.

However, I ran into a little snag while editing and publishing everything. The problem was that each photo’s GPS/location data wasn’t making the jump to Flickr from Aperture. The solution was these two check boxes that needed to be selected in order for the location info to be correctly exported: one in Advanced Options and another in Export Preferences.

Aperture Advanced Options

Aperture Export Preferences

So in the end, if you make sure “Include location information for published photos” and “Include location info in exported photos” are both selected, everything works beautifully.

Aperture Places view

Aperture Places

Flickr Map view

Flickr Map

Lid Open, Display Off

It has been more than a year since my last post and that computer has been retired. However, I still find it useful to run my new MacBook Air attached to an external monitor and with its internal display disabled. A few days ago, I found a better and more permanent solution than a flimsy magnet:

  1. Attach the closed MacBook to an external display
  2. Wake the MacBook with an external input device, like usb or bluetooth keyboard.
  3. Open the lid of the MacBook and the internal display will remain off

That’s it. No magnets required.

It takes a little bit more for this to work under OS X 10.7 Lion, though. Entering the following command in terminal seems to do the trick:

``` shell sudo nvram boot-args=“iog=0x0”

Undoing this is as simple as typing the following, also in terminal:

``` shell
sudo nvram -d boot-args 

You can also zap the PRAM (press Cmd+Opt+p+r at power up) to restore it to the new Lion behaviour.

This works great and I’ve been using it to improve my laptop’s ventilation and wireless reception.

I first saw this documented here: Mac OS X Hints - 10.7: Disable internal laptop display when external display is attached

Trick a Macbook into thinking it is closed

UPDATED Nov 2011: I found a better way to do this that is posted here

Up until now, I have run my laptop at work with the lid closed and everything on an external monitor on my desk. However, I was starting to get really concerned about the heat (60C idle and 95C under load) and how the little fan was struggling to keep up.

So, I started looking around for tips about how to leave the lid open, but trick my MacBook into thinking the lid was closed and leave the LCD off. I found a few good tips, but I still had to hunt with a magnet for that magical switch point. Well, I found it and the picture is below. Things look pretty frosty so far, let’s hope this extends the life of my 3 year old laptop a bit more.

MacBook with lid-closed magnet
applied

iTunes library at home accessible over SSH tunneled AFP

Today I freed 150GB of space from my Macbook by moving my music to my Mac Mini that sits below my TV, at home, behind two firewalls.

First, I hardened SSH on the mini and setup my laptop to log in without a password, but I didn’t go through the SSHFS stuff, as I like AFP better (for the moment). Now, I can access my Mini by running:

``` shell LAPTOP$ ssh -f -N -p PORT USER@OUTSIDEIP -L 1202:localhost:5900 -L 1203:localhost:548

Where PORT is the random port I choose when [hardening SSH][1], USER is the
username on my mini and OUTSIDEIP is the external IP of my cable modem.  Then,
I can exit from Terminal on my laptop, press command-K and run:

``` shell
afp://localhost:1203

and get the list of shared folders. Or I can run:

``` shell vnc://localhost:1202

and get Screen Sharing with the Mini behind all those firewalls.

Finally, I [moved my iTunes Library to an external harddrive] [2] and hooked up
that beast to the Mini.  By SSHing into my Mini and then running the AFP
command, I can now access my very large iTunes library from anywhere with an
internet connection.

If I would like to end the SSH tunnel, I run:

``` shell
ps auxww | grep -i ssh

After finding the ID of the process I do:

shell kill -9 SSH_PID `

with the SSH ID.

I will use some applescript to make this “connect, mount, launch iTunes” dance a little bit more simple, but I think this is progress, as it has breathed new life into my 2.5 year-old laptop.

Automatically upload new files to Google Documents from OSX

##The Problem

I get many, many, many presentations, spreadsheets and PDFs sent to me over email. I usually View them through Google Docs or Download them and let the files launch in their proper programs. There are some times, however, when I wish I could view or edit these files on my iPhone or on another computer, usually located in our cleanroom. Dropbox is an incredible service for this, with an iPhone app and multi-platform support, but those fab computers are locked down hard and I can’t edit things in the iPhone app.

##The Caveat

Since I use Gmail for everything, I could just view the file through the web interface on my iPhone or another computer and import things to Google Documents with a click.

I recommend this way. I really do. Mostly because my solution is a folder action applescript with your gmail password in plain text. This is stupid. Very stupid. Beyond moronic. I would like to get Keychain Access from the applescript for a better password handling, but a few hours of fiddling yielded nothing and further thought showed it to be even more insecure to have scriptable readouts of passwords globally enabled.

So, for the record, this is my hacked together solution.

##The Solution

  1. Download google-docs-upload and put it somewhere. I used /Users/USERNAME/Library/Scripts/GoogleDocs/

  2. Open Script Editor.app and it should pop up with an Untitled and blank script window. Copy this into the window:

applescript on adding folder items to this_folder after receiving added_items repeat with aFile in added_items do shell script "java -jar /SOMEWHERE/google-docs-upload-1.3.2.jar aFile -rf Downloads --skip-all -u YOURUSERNAME -p YOURPASSWORD >> /SOMEWHERE/GDocs-upload-log.txt" end repeat end adding folder items to `

  1. Replace SOMEWHERE with the path to google-docs-upload, USERNAME with your OSX username, YOURUSERNAME with your Google username and YOURPASSWORD with your google password. This script will run when a file is placed in the folder. Each file is passed to google-docs-upload along with your username and password and it decides whether it can upload the file. Any output goes to that log file tacked on the end.

  2. Save this script under ./Library/Scripts/Folder\ Action
    Scripts/UploadGDocs.scpt

  3. Right click on your Downloads folder and hover over “More” until it expands. Go and click on “Configure Folder Actions…”

  4. Make sure the “Enable Folder Actions” check box is ticked, then click on the left “+” sign, select the Downloads folder and click the Open button. Then, select UploadGDocs.scpt from the next menu to drop down and click Attach. Close the Script Editor and the Folder Actions Setup.

That should do it. Any new file placed in your Downloads folder will be kicked automatically to Google Docs.

PS: For some reason, whenever I tweak this script, my Downloads folder becomes Read-Only for my user. I fix this by right clicking on my Downloads folder, clicking on Get Info, clicking the lock icon in the new window, inputting my password and then setting the drop-down menu next to my name to Read&Write. If you know what is causing this, let me know.